AD-A075  578  STANFORD  UNIV  CA  DEPT  OF  OPERATIONS  RESEARCH  F/G  12/1 

PROPERTIES  OF  ISOTONIC  ESTIMATORS  OF  MEAN  LIFETIME  IN  A  SIMPLE  — ETC(U) 
AUG  79  T  P  MCWILLIAMS  N00014-75-C-0561 

UNCLASSIFIED  TR-194  NL 


PROPERTIES  OP  JSOTONIC  ESTIMATORS  OP  JJEAN  LIFETIME 
IN  A  $IMPLE  PROTOTYPE  DEVELOPMENT  MODEL, 


thomas  p.  McWilliams 


TECHNICAL  REPORT  NO.  194 


SUPPORTED  m  ARMY  AND  NAVY 
UNDER  CONTRACT  SOOOI4-75-C-OH1  (NR-04 7-200) 
WITH  THE  OftitEOr  NAVAL  RESEARCH 


Gerald  J.  Lirbrrswn,  Project  Director 


Reproduction  in  Whole  or  in  Part  la  Permitted 
for  any  Purpose  of  the  United  States  Government 

Approved  for  public  release;  distribution  unlimited 


D  D  C 

I®EIME 

OCT  29  1979 


ODC  TAB 
Uhnonaunesd 
Just  If loatlon 


DEPARTMENT  OF  OPERATIONS  RESEARCH 
AND 

DEPARTMENT  OF  STATISTICS 
STANFORD  UNIVERSITY 
STANFORD,  CALIFORNIA 


r lbutla 


Avail  and/or 
special 


ACKNOWLEDGMENTS 


1  would  like  to  thank  ay  advisor.  Dr.  Cerald  J.  Lleberaan, 
for  hla  unflagging  support  and  enthusiasm;  his  patience  during 
periods  of  scholastic  adversity  made  this  dissertation  possible. 

I  would  also  like  to  thank  Dr.  Olkln  and  Dr.  Slngpurwalla  for 
their  suggestions,  cooaents,  and  service  on  the  reading  cuaalttee. 
In  addition,  thanks  to  Audrey  Stevenln  for  an  excellent  and  very 
patient  Job  of  typing. 

A  special  thanks  to  ay  wife,  Connie,  for  her  support  and 
understanding  during  our  years  at  Stanford. 


11 


ABSTRACT 


Consideration  of  a  simple  prototype  development/rellablllty 

growth  model  leads  to  observation  of  variables  X^,  . X^; 

where  the  X (  are  independent  and  are  assumed  to  have  the  same 
distribution  type  with  Individual  means  1^.  The  problem  Is  to 
estimate  the  A^,  which  are  aasusvd  to  be  non-decreasing  due  to 
design  improvements  in  the  device  under  development.  For  many 
common  distributions,  estimation  via  the  restricted  maximum  likeli¬ 
hood  principle  leads  to  estimates  which  arc  the  Isotonic  regression 
of  the  Xj  with  appropriate  weights.  This  paper  finds  the 

A 

distribution  of  A^,  the  mlc  of  A^,  when  the  X^  are  exponentially 

A 

distributed.  A^  is  then  compared  with  X  and  X^,  two  competing 
estimates  of  A^.  Camma,  normal,  and  Welbull  distributed  X^  are 

A 

also  considered,  and  the  usefulness  of  A^  Is  seen  to  depend  on  the 
relative  magnitude  of  the  variances  and  means  of  the  X^ .  Results 
are  also  examined  when  the  restricted  rnlrs  are  generated  assuming  the 
A^  are  a  non-decreasing  and  concave  function  of  i. 
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CHAPTER  1 


INTRODUCTION 


1.1.  The  Model 


Consider  a  sequence  of  Independent  random  variables 


The  X  are  assumed  to  have  distributions  which 


belong  to  the  same  family,  with  the  possibility  of  unequal  means 


Let  A.  denote  the  mean  of  X. ,  i  •  l 


unknown,  but  we  assume  the  ordering  relation  A  A  < 


holds.  The  problem  treated  in  this  dissertation  is  that  of 


efficiently  estimating  the  A  under  various  distributional 


assumptions.  A  look  at  the  practical  considerations  from  which 


the  model  was  developed  indicates  that  the  estimation  of  A  the 


final  mean  in  the  sequence,  is  of  particular  interest.  The  case  where 


the  X  are  exponentially  distributed  will  be  the  most  thoroughly 


examined.  We  will  also  examine  the  cases  where  the  X 


normal,  and  Wclbull  distributions 


The  model  described  above  arose  from  the  following  situation 


Consider  a  prototype  development/testlng  procedure  where  a  device 


prototype  is  tested  and  its  lifetime  X  is  observed.  When  the 


prototype  falls,  the  cause  of  failure  is  determined  and  a  new 


prototype  is  designed  in  an  attempt  to  eliminate  or  reduce  the 


probability  of  this  type  of  failure.  This  new  prototype  is  tested 


and  the  process  continued,  yielding  a  sequence  of  observed  lifetime 


If  the  basic  design  and  construction  of 


_ 


the  prototypes  Is  unchanged  throughout  the  development  process,  then 
the  assumption  that  the  have  the  same  distribution  type  would 

not  be  unreasonable.  In  addition,  if  we  choose  to  believe  that  the 
design  changes  at  each  stage  are  Indeed  improvements,  then  the 
assumption  that  the  A^  are  non-decreasing  is  a  natural  one.  The 
focus  of  Interest  on  the  final  mean,  A^,  is  due  to  the  fact  that 
this  value  represents  the  current  state  of  the  development  process. 

A  major  portion  of  this  paper  is  devoted  to  the  evaluation  of  various 
estimates  of  A^  in  certain  situations,  and  some  work  is  done  on 
testing  hypotheses  to  determine  whether  or  not  a  desired  mean  life¬ 
time  has  been  achieved  in  the  development  process. 

Note  that  nothing  has  been  said  about  the  nature  of  the  change 
in  mean  lifetime  at  each  stage,  beyond  the  assumption  that  it  is  non- 
negative.  For  the  bulk  of  the  paper  thi#  will  be  the  oniy  assumption 
made  -  no  particular  functional  relationship  among  the  A^  will  be 
considered.  The  only  further  restricting  assumption  will  be  in 
consideration  of  the  case  where  the  arc  not  only  non-decreasing 

but  are  also  a  concave  function  of  1, 

1.2.  Maximum  Likelihood  Estl ma t Ion 

One  approach  to  the  parameter  estimation  problem  described  in 
the  preceding  section  is  the  use  of  the  maximum  likelihood  principle. 

A  A 

Maximum  likelihood  estimates  are  given  by  the  values  A^ . Afc 

which  maximise  the  Joint  density  function  of  the  X^.  Since  the  Xj 
are  independent  and  we  have  assumed  that  their  distribution  functions 


2 


belong  to  the  sue  family,  we  obtain  for  the  Joint  density  function 
the  simple  expression: 


k 

L(v)  -  R  r  (x  ;  v.)  (1.2.1) 

i-1 

where  f^x^;  represents  the  density  function  of  Xj .  The 

assumption  of  non-decreasing  A^  should  naturally  be  taken  into 
consideration,  and  consequently  we  arrive  at  the  problem  of  maxlmiz- 

A  A 

ing  expression  (1.2.1)  under  the  assumption  A^  <_  •••  <_  A^.  Note 

that  v^  represents  the  entire  parameter  vector,  while  A^  denotes 

the  mean  of  the  distribution.  Thus,  the  value  of  A^  may  Itself  be 

an  clement  of  v, ,  or  It  may  be  a  function  of  one  or  more  of  the 
1 

elements  of  . 

For  many  densities,  the  restricted  maximization  problem 
described  above  Is  difficult  and  time-consuming,  and  no  closed  form 
expression  for  the  A^  can  be  found.  Fortunately,  however,  there  are 
many  Important  cases  where  this  problem  has  been  shown  to  be  equivalent 
to  that  of  finding  an  isotonic  regression  of  the  with  appropriate 

weights.  This  equivalence  holds  for  a  fairly  broad  exponential  class 
of  density  functlona,  which  Includes  the  normal  (with  known  variance), 
gamma,  binomial,  and  Poisson  densities.  In  these  cases,  the  values 
which  give  the  isotonic  regression  of  the  X^  are  equal  to  the  values 
which  maximize  the  likelihood  function  under  the  non-decreasing 
constraint . 


3 


Section  2.2  includes  a  proof  of  tlie  sforement ioned  equivalence 


In  the  case  of  exponentially  distributed  .  Similar  results  for 
the  other  parent  distributions  listed  above  can  be  found  in  Chapter  2 
of  Barlow,  Bartholomew,  Breoner,  and  Brunk  1 1 ) •  Isotonic  regression 
is  defined  in  Section  1.3,  and  computat ional  algorithms  are  given  in 
Sections  1.3  and  3.2. 

1.3.  Isotonic  Regression 

The  isotonic  regression  of  ,  . . . ,  X^  with  weights 

*  * 

w1 . wfc  is  defined  to  be  the  set  of  values  X^ . X^  which 

minimize  the  weighted  sum  of  squared  deviations 

k  , 

[  (Y  -  X  )  w  (1.3.1) 

1-1  1  1 

where  the  minimum  is  taken  over  all  values  Y^ . such  that 

Y^  <  Y,  •••  <_  Y^.  There  are  several  relatively  simple  methods  for 

*  „ 
computing  the  X^.  The  primary  method  used  in  this  paper  is  the  mln- 

max"  formula  (Barlow,  Bartholomew,  Bremner,  and  Brunk,  page  19  II)), 

given  by 


a 


t 

max  min  ! 
s<i  tM  r-s 


t 


r-s 


(1.3.2) 


This  formula  is  very  easy  to  use.  Involving  only  the  calculation  and 

comparison  of  various  weighted  averages  of  the  data.  In  subsequent 

* 

sections,  we  will  consider  parent  distributions  for  which  the  X{ 
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(computed  using  unit  weights)  will  prove  to  be  maximum  likelihood 

* 

estimates  of  the  X^,  and  consequently  the  Xj  will  be  denoted 

A 

X^  for  the  remainder  of  the  paper.  Note  that  if  we  consider 
estimation  of  the  final  mean,  and  If  all  observations  are 

equally  weighted,  formula  (1.3.2)  can  be  reduced  to: 


A  K 

l  x  /(k-s+1) 

*  r-s  r 


L  Wi  VV1+V2 

|V  2  •  3 


...  X 


(1.3.3) 


where  X  represents  the  overall  mean  of  the  X^.  For  evaluation 
purposes,  it  would  be  desirable  to  obtain  the  distributions  of  the 

A 

X.  for  a  variety  of  parent  distributions  f„(x, ;  v . ) .  This  can  be 

1  A  1  — 1 

very  difficult  even  when  k,  the  number  of  stages  in  the  process,  is 
small  and  the  density  function  f  is  fairly  simple.  Due  to  its 
computationally  easier  form,  some  progress  has  been  made  on  finding 
the  distribution  of  X^.  In  a  paper  by  David  Williams  (1977),  the 

A  A 

distribution  of  X^  and  (which  also  has  a  computationally  easier 

*  1/ 

form  than  the  remaining  X^  — '  )  is  found  in  the  case  where  the  X^ 
are  Independent  N(0,1)  random  variables.  No  distributions  are 
found  in  the  case  where  the  means  X^  are  not  equal,  but  the  equal- 
mean  distribution  is  used  to  test  the  hypothesis  that  the  X^  are 
Indeed  equal  versus  the  alternative  of  unequal  non-decreasing  means. 


■InJxj,  j 


X1*X2  X!+X2*X3 


x! 
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Hie  problem  of  determining  point  estimates  of  the  individual  X^ 

In  the  case  of  non-decreasing  means  Is  not  addressed. 

This  paper  will  develop  a  formula  which  can  be  used  to  find 

A 

the  distribution  of  X^  in  the  case  where  the  are  Independent 

and  exponentially  distributed  with  possibly  unequal  means.  An 
exact  closed-form  distribution  will  be  given  In  the  equal-mean  case, 
and  hypothesis  testing  and  asymptotics  will  be  discussed.  Other 
distributional  assumptions  will  also  be  considered,  with  results 
primarily  obtained  through  simulation.  In  addition,  a  "concave 
Isotonic"  technique  will  be  examined,  where  the  X^  are  assumed 
to  be  non-decreasing  and  to  be  a  concave  function  of  1. 
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Y 


1 


Y 


2 


V  - 

k 

V  — 

k-1 


Y 


k-1 


VA 

2 


Y 


k 


-  s 


which  gives:  F^tx)  “  MYj  1  *•  *2  -  * . Yk  -  * 

Solving  for  the  X ;  In  the  above  systea  of  equations  we 
obtain,  for  the  Inverse  t ran* format  Ion: 


-  kYL  -  (k-l)Y2 
Xt  -  (k-l+l)Yt  -  (k-i)Y1H 


*k-l  '  ‘'k-1  “  \ 
Xk-Yk  . 


The  Jacobian  of  this  transformat  Ion  Is  easily  seen  to  be  kl,  and  the 
Joint  density  of  the  Y^  can  be  expressed: 


k 

(y  .  ....  y.)  -  k!  n  f  ((k-1+1)  y. 
‘'k  1  *  1«1  *  1 


(k-l)  y14l;  vq) 


8 


Then  L(A;  x)  -  max  L(_A;  x)  if  and  only  If  A  la  the 
A 

Isotonic  regression  of  x  with  unit  weights. 

Proofs  First  note  that  maximizing  the  likelihood  function  L(X^;  x) 

Is  equivalent  to  minimizing  the  negative  log  likelihood  function, 
given  by 

k 

t(X;  x)  -  l  (log  A  +  x./A  )  • 

1-1  iii 

How  let  Mu)  -  -log  u  and  define  A*(u,v)  -  Mu)  -  *(v)  -  (u-v)Mv) 
where  Mu)  -  #'(u)  -  “  1/u.  Then 

Vv  V  *  ^  (" 108  xi + 108  \  *  (*rAi)  <- 

k  k 

“  -  [  log  x  ♦  [  (log  1  ♦  x  /A  )  "  k  • 

1-1  1-1  1  11 

This  expression  differs  from  the  negative  log  likelihood  function 
only  In  terms  which  do  not  Include  A_,  so  minimizing  MM  x)  Is 

seen  to  be  equivalent  to  minimizing  A. (x. ,  A  )  . 

1-1  1  1 

Now  apply  theorem  1.10,  page  41  of  Barlow,  Bartholomew, 
Bremner,  and  Brunk  (l)  .  This  theorem,  rewritten  using  our 
notation,  states  that  If  A^  c  A  then 

k  k  a  ^  * 

[  A#(x  .  *.)  >  l  A#(*t.  *t>  ♦  l  ^(*1*  At)  (2.2.2) 

l-l  1-1  1-1 


11 


* 

where  x  Is  the  isotonic  recession  of  the  Xj  (with  unit  weights) 
and  *  is  an  arbitrary  convex  function.  Note  that  since  *  la 
assumed  to  be  convex,  A#(u,  v)  is  easily  seen  to  be  non-negative. 
Using  this  fact  and  applying  expression  (2.2.2)  with  8(u)  •  -  log  u 
(which  Is  Indeed  convex)  we  obtain  the  result: 

k  k  , 

l  Vv  V  -  £  W  V*  VAcA. 

1-1  1  1-1 

•  k 

So,  j  Is  seen  to  minimize  \  A.fx.,  A.)  over  A  and  consequently 

1-1  1  1 

e 

x  minimizes  l (A,  x)  over  A  .  Therefore  the  Isotonic  regression 

*  *r-i 

x  and  the  maximum  likelihood  estimate  A  are  one  and  the  same.  □ 


Now  substitute  the  exponential  density  function  Into  expression 
(2.1.2)  to  find  the  distribution  of  the  isotonic  or  maximum  likelihood 


estimate  of  A^: 


XX  X 

P.U)  •  /  /  •••  /  k!  8  6  •••  8. 

0  yk/2  I(k-l)/k)y2  1  2  k 


*  •*p(-l«1(ky1  -  (k-1)  y2)  ♦  e2 (( k-1  >  y2  -  (k-2)  y3)  ♦  •••  ♦  6 


•  dyj  dy2  •••  dyk  . 


(2.2.3) 


Combining  the  terms  in  the  exponent  which  have  common  factors,  we 


XX  X 

F  (x)  -  /  /  •••  / 

0  yk/2  I <k-l)/k)y2 


k,el62  °k 


•  exp(-(k6  ^y1  ♦  (k-l)(e2-«1)  y2  ♦  •••  ♦  (6^6^)  yk  ] ) 


dyl  dy2  **•  dFk  • 


(2.2.4) 


This  Integral  Is  difficult  to  evaluate  In  closed  fora,  but  a  recursive 
relationship  has  been  developed  which  can  be  used  to  determine  K^Cx) 
for  any  value  of  k. 


Theorem  2.2:  Denote  the  Integral  given  In  expression  (2.2.4)  by 
a^Gj,  . ...  6^)  (consider  x  to  be  fixed).  Then 

-kP.x 

*  e2  •••  efc 

V9i . V  *  Vi(e2 . V  "  <°2-«1)  —  (°k“e1) 


*  •k.l (®2-®i *  ••••  ®k”®l^  * 


(2.2.5) 


Proof;  Evaluating  the  Innermost  Integral  of  *^(11)  gives 


xxx  f  -(k-l)».y?  -kd.x 

-  (8)  -  /  /  •••  /  (k-l)l  e,  •••  6  [e  1  -  e 

0  yk/2  l€k-2)/(k-l) Jy3  ( 


•  exp{-f  (k-l)(e2-ei)  y  2  -f  •••  ♦  <ek~ek-l)  yk^  dy2  ”*  dyk  * 


Separate  this  Into  two  Integrals  to  obtain 
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I 


X  X 


/  •••  / 

0  1  (k-2)/(k-l) )y . 


(k-1)!  02  •••  9k 


•  expM(k-l)  e2y2  (k-2)(63-e2)  y}  ♦  •••  ♦  (Ok-ek_l)  ykJ) 


-ke  i  x  x 

dy,  «iyfc  -  *  /  •**  / 

2  k  0  I(k-2)/(k-l)Jy3 


(k-1)!  e2  •••  ek 


**p(“  I  (k-1)  (#2  03)  y2  ♦  •••  ♦  (V9k-1}  yk,}  dy2  ***  dyk 


The  flrat  Integral  la  aeen  to  be  equal  to  ak  ^  (©2 . 6^) . 


Substituting  this  In  and  Multiplying  by  the  appropriate  factora,  we 


-k9.x 

.  e2  —  ek 


Vei . V  “  Vi(e2 . ek)  "  («2-e1i(e3-e1)  ••• 


/  —  / 


0  [(k-2)/(k-l)Jy, 


(k-1)!  <«2‘V<93~V  •••  (ek-*j) 


•  exp|-( (k-l)(02-ei)  y2  ♦  ...  ♦  <ek-«k_1)  yk))  dy2  •••  dyfc 


The  remaining  Integral  la  aeen  to  be  equal  to  C3~ei . 9k~9l^ 

Expression  (2.2.S)  la  obtained  by  direct  aubatitutlon.  D 


The  relatlonahlp  given  In  Theorea  2.2  can  be  uaed  to  find  the 


diatrlbutlon  function  of  Xk  for  aucceaalve  valuea  of  k  .  Flrat 


conalder  the  trivial  cast,  when  k  •  1: 


»  -ex 

a  i  (®  i )  ”  <  »)  ■  P(  aax(X^)  <  i)  •  1-e 


1 


When  k  -  2,  expression  (2.2.5)  Is  used  to  obtsln 


'2(V  V 


-20^ 

*1(02)  -  *l(e2~®l) 


1  -  e 


-9  x  9,  '2°1X  /  -(92_e,)x 

2  2  e 


V*I 


’lx  ^.e‘  2  1  ) 


-92x  92  I"  -20  *  -(61+«2)x1 

"  1  -  *  -  o^T  l-  *  *  J 
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Appendix  A  gives  expressions  for  the  distribution  function  when 
k  -  3  end  k  -  4.  This  rather  tedious  process  can  be  continued  to 

A 

find  the  distribution  function  of  A^  for  any  value  of  k.  Note 
that  the  problem  of  division  by  zero  la  encountered  above  when 

-  ®2 ,  and  this  will  continue  to  be  a  problen  for  larger  values  of 
k  whenever  any  pair  of  the  0^  are  equal.  An  exact  expression  for 

A 

the  distribution  of  A^  when  all  of  the  0^  are  equal  will  be  found 
in  the  next  section.  In  Appendix  B  it  la  shown  that  the  distributions 
for  the  "in-between"  cases  where  at  least  one  pair  of  the  0j  are 
equal  can  be  found  by  computing  a^(0)  via  the  recursive  technique 
and  then  taking  the  appropriate  limits. 

A 

Once  the  distribution  function  of  Afc  has  been  found,  its  mean, 

variance,  and  any  other  desired  moments  are  easily  computed.  Recalling 
-0.x 

that  Fj(x)  -  1  -  e  and  considering  the  nature  of  the  recursive 

relation  given  in  expression  (2.2.5),  it  is  seen  that  the  distribution 

A 

function  of  A^  will  always  be  of  the  form 


Fk(x) 


2-1 

l 

1-1 


(2.2.6) 


15 


for  som  real  coefficient*  end  0^.  This  represents  s  mixture 

of  exponential  density  functions.  Now  consider  the  moment  equation 


16 


The  distribution  function  F,(x)  is  seen  to  fit  the  fora  of  expression 
(2.2.6)  with 


«,  - 1 


»1  '  *2 


“2  '  V<e2  -  V  S2-29l 


°3  '  -  V<92  -  V  *1  '  91  *  9 2 


Consequently,  expression  (2.2.8)  can  be  used  to  obtain  the  moments 


*12 
E<*,)  “  T-  * 


2  02  HjWj-lj)  (e1-*^2)(e2-e1) 


E(X2‘)  -  2 


_L  + _ 

,2  ,a2 


e;  (0,+0,)‘(0,'0,) 


1  27  '  2  1 


Other  moments  can  be  computed  in  a  similar  manner.  The  expected  value 

A 

of  for  the  cases  k  •  3  and  k  ■  4  is  given  in  Appendix  A. 


2.3.  Underlying  Exponential  Distributions  with  Equal  Means 

When  the  X^  are  all  equal  to  a  common  value  X  -  1/0,  expression 
(2.2.4)  simplifies  considerably  and  a  closed  form  solution  can  be  found 
for  any  value  of  k.  The  equal  X^  case  will  prove  useful  for 
(1)  testing  hypotheses  regarding  1^;  (2)  evaluating  the  asymptotic 

A  A 

distribution  of  X  as  k  ■*  •;  and  (3)  comparing  X.  with  other 


estimates  of  X^  .  These  topics  will  be  discussed  in  detail  in 
Sections  2.4  -  2.7. 


Theorem  2.3:  Let  X^,  X^,  . . . ,  X^  be  Independent  exponentially 
distributed  lifetimes  with  common  mean  A  -  1/6.  Let 


r  /„  Wi  -  \ 

\  “*\V  2  X) 


Then  the  COF  of  X^  Is  given  by 


k  \t-l  -J»* 

V  J  (6x)  c  J 


F  (x)  -  1  -  ?  ^ - — 

k  jti  (J-D« 


Proof:  Replace  each  6^  by  6  In  expression  (2.2.4)  to  obtain 


Fk(x) 


XX  x  -key 

//  **•/  k!  6  c  dy  dy  •••  dy 

0  yk/2  |(k-l)/kjy2  1  2  k 


Consider  x  fixed  and  denote  this  Integral  bk(6).  Evaluating  the 
innermost  Integral  gives 


bk(6) 


/  I 

o  y, 


X 

/  (k-l)l 

I(k-2)/(k-l))y3 


-l)ey. 


•  dy2 


dyk 
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(k-i)J  (r+i+l) !  krk,J 

where  k  ^  2  and  r  ^  0  .  A  proof  of  Lemma  1  can  be  found  in  Appendix  B. 

Now  make  the  substitution  •  y^/x  for  1*2,  3,...tk  In 
expression  (2.3.2)  to  obtain 


r+1 


( (k— 1 ) ! 


L 

1-1 


a  •  x 
k 


k-l 


1  1 
I  J 


*k/2 


/ 

((k-2)/(k-l))x. 


dz  , 


dr, 


-  xk_1  c(0) 
k-l 


From  Lemma  1, 


c<°>  -  +  V  (_ni 

Ck-1  ?  Z  Vi;  (k-l-D! 

“  1  ( (k-2) ! )*  1-1  '  1  1,1 


(i«-l>!  (k-l)  J 


,vk-3  k-2  .  .  .  .  k-l 

-  ♦  [  (-l)k"1_J 


1 


t (k-2) ! )  j-1 


j!  (k-J) ! (k-l) ! 


(let  j  -  k-l-i) 


k;l  .  .k-l 

l  (-1)“  1J  ± 

J-1 


1 


J!  (k-J) ! (k-l) ! 


(Incorporate  the  first  term  Into  the  sum) 


k-l  k 

k!(k“lTT 


.  tAl 


(-1) 


k! 


.k-l 


k-2 


Jt(k-J)! 


I (k-l) !  )' 


(add  and  subtract  the  J  -  0  and  J  -  k  terms) 


The  auaucton  given  above  la  equal  to  aero  by  a  roablnatorlal 
Identity  (feller,  Vol.  I,  page  65  l 8 1 ) -  Conaequcnt ly  we  have 


k-2 


( (k-1) I J 


2 


and 


% 


XX  X 

II  •••  /  dfj  •••  dy. 

0  yk/2  [ (k-2)/(k-l) ly j 


k-1  .k-2 
x  k 

l(k-l)Jj2 


Subetltute  this  reault  Into  expreaalon  (2.3.1)  to  obtain 


V9>  -  bk-i(9>  - 


kk~2(8x)k"yke* 

(k-1)! 


or 


bk(e)  -  bk_l(e) 


.k-2..  .k-1  -k8x 
*  (•*)  '  • 
(k-l)l 


(2.3.3) 


Now 


bk(9)  -  b^e)  ♦ 


I 


0^(9)  -  b  ^(8)) 


(2.3.*) 


and 


b^S) 


dyl 


1  - 


-8x 

a 


(2.3.5) 
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so  we  have,  combining  expressions  (2.2.3),  (2.3.4),  and  (2.3.5) 


bk(6)  -  1  -  e 


)  jA-WV»» 

jii  w 


r  J“1  ”J6* 


,  .  y  l  -  iLKJL-  j» 
JM  <>-»' 


(2.3.6) 


This  concludes  the  proof  of  Theorem  2.3.  □ 


Corollary:  The  density  function  of  A^  In  the  equal  A  case  Is 
given  by 

*kU)  "  ^  TwTT  •  -  O-Dl  • 

The  corollary  Is  obtained  by  differentiating  expression  (2.3.6)  and 
combining  terms. 

We  now  have  expressions  for  the  distribution  and  density  functions 

A 

of  »k  in  the  equal  A  case  for  any  value  of  k.  Graphs  of  the  denalty 
functions  for  A  •  10  and  values  of  k  ranging  from  1  to  100  are 
given  in  Figures  l  and  2.  Note  that  the  densities  have  a  heavy  right- 

A 

hand  tall,  which  will  be  seen  to  lead  to  high  variances  for  the  A^. 

A 

The  moments  of  A^  are  easily  found  using  expression  (2.2.7) 
with  F^(x)  substituted  for  F(x): 
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l 


10  v 


1 


Figure  2.  Density  of  A  ^  In  the  Case  of  Equal  A^. 
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Evaluating  mean-square-er ror  (HSE),  which  will  be  used  In  the  future 
to  evaluate  the  effectiveness  of  as  an  eatlskator,  gives  the 

Interesting  result 

HSE(Xk)  -  2X2  n(k)  -  x2  n2(k)  ♦  (Xn(k)  -  X)2 

-  2X2  n(k)  -  X2  n2(k)  ♦  X2  n2(k)  -  2X2  n(k)  ♦  X2  -  X 

So,  In  the  case  when  the  X^  are  all  equal,  the  HSE  of  X^  Is 
Independent  of  k.  the  number  of  stages. 


2.4.  Efficiency  of  Isotonic  Eatlsutors  of  the  1^ 

A 

The  Xt  are  Intuitively  attractive  estimators  of  the  mean 
lifetimes  for  two  reasons.  Firstly,  they  take  the  ordering  of  the  X^ 
Into  account  by  maximising  the  likelihood  function  over  the  restricted 

AAA  A 

space  1^  i  Xj  <,  *•*  £  X^  .  Secondly,  each  1^  Is  a  function  of  aH 
of  the  random  variables  Xj,  X^,  ...,  X^  and  consequently  each 
estimate  makes  use  of  all  of  the  Information  available.  This  la 
desirable,  for  example,  because  although  X^  does  not  have  mean  X^, 
Its  mean  Is  related  to  X^  through  the  ordering  relationship,  and 
consequently  Xj  may  contain  useful  Information  regarding  X  . 

We  wish  to  evaluate  the  perforsunce  of  the  isotonic  estimates 
In  the  case  of  exponentially  distributed  Xj.  The  criteria  for 
evaluation  will  be  mean-square-error,  a  coimson  measure  of  estlsutor 
effectiveness.  As  a  basis  for  comparison,  two  competing  estimates  of 
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the  A^  are  proposed.  The  first  Is  to  use  X,  the  overall  sean  of 
the  X^,  as  a  common  estimate  of  the  X  would  he  an  appropriate 

estimate  In  the  boundary  case  where  the  A^  are  all  equal  (which 
makes  the  X^  lid  randoa  variables).  In  fact,  X  Is  easily  seen 
to  be  the  uniform  minima  variance  unbiased  estimate  (UMVUE)  In  this 
situation.  At  the  beginning  of  the  Investigation,  It  was  expected 
that  X  would  perform  comparatively  well  when  the  A^  were  equal  or 
close  to  equal,  but  would  not  do  as  well  when  the  A^  varied  widely. 

The  second  proposed  "competitor"  of  the  A^  Is  simply  to  use 
X^  as  the  estimate  of  A^  at  each  stage.  In  effect,  we  treat  the 

observations  X^,  X^,  ....  . X^  as  If  they  are  totally 

unrelated  to  A^  and  consequently  are  of  no  value  In  estimating  A^. 
It  Is  Interesting  to  note  that  If  the  A^  are  widely  spaced  (for 

|  | 

example.  If  min  !  — —  ,  Is  approaching  •) ,  these  estimates  are 

'  1  ‘  1-1 

effectively  equal  to  the  Isotonic  regression  estimates,  since  the 
X j  are  themselves  non-decreasing  with  high  probability.  Ue  would 
expect  that  the  X^  estimates  will  do  comparatively  well  when  the  A^ 
are  Indeed  widely  spaced,  but  will  not  fare  as  well  In  other  cases. 

To  compare  the  three  proposed  methods  of  estimation,  two  cases 

♦ 

are  examined.  The  first  Is  total  maan-aquare-error  (MSE)  over  all 
the  A ^ .  In  this  case,  the  MSE  for  the  X  and  X^  estimates  are 
computed  exactly  while,  due  to  computational  difficulty,  the  MSE's 

A 

for  the  A  are  found  via  computer  simulation.  The  second  case  is  to 

restrict  our  Investigation  to  the  performance  of  estimators  of  the 


27 


final  mean  A^  :  A^,  X,  and  X^.  Since  we  have  developed  a  recursive 

A 

technique  for  finding  the  distribution  and  moments  of  A^  in  the 
exponential  case,  all  computations  are  exact  and  no  simulation 
studies  are  required  here. 

The  computation  of  MSE  for  the  X  and  X}  estimates  is 
straightforward.  For  X  we  have: 


E(X) 


k 

l  * 

J-l 


J 


l  < 


and  Var(X)  - 


j-l 


So.  MSE^X) 


-  Var(X)  ♦  Blasf(X) 


(2.4.1) 


where  NSE^(X)  and  Blas^(X)  represent  the  mean-square-error  and 
bias  when  X  is  used  to  estimate  Aj.  For  Xt  (which  is  unbiased 
for  A vr  have: 


MSE (X t >  -  Var(Xt)  -  A*  . 


A  computer  program  is  used  to  compute  the  exact  moments  and 

A 

for  A^.  Let  us  examine  the  fairly  manageable  case  k  -  2. 

*  '2  * 

E(*2)  *nd  E(A2)  were  given  in  Section  2.2,  and  EfA^)  can  be 
rewritten: 
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*(X2>  '  t 


26l(ei+62) 


Since  9j  end  0  ,  ere  both  positive,  this  fore  of  expression 

A 

cleerly  shows  the  negative  bias  or  tendency  of  to  overest lmete. 

This  is  true  in  general  for  all  values  of  k,  since 


-  ~x(V 


V*k-1 


and  X^  Is  unbiased  for  A^.  Note  that  A^  has  a  positive 

probability  of  being  strictly  greater  than  In  all  but  the  trivial 

case  X  -  X  -  • • •  -  A,  -  0  .  Now 
12  k-1 


MSE(A2)  -  Var(X2)  ♦  Rlas2(X2> 

-  E(A2)  -  E(A2)2  (E(X2)  -  X2)2 

-  E(xj)  -  2Xj  E(X2)  ♦  X2  . 


Substituting  the  previously  derived  expressions  for  E(X2>  and 
E(*2),  we  obtain,  after  some  algebraic  manipulation: 


Letting  k 
we  obtain 


NSE(X2) 


(02-01)(201+«2) 

26l(ei+62)2 


(2.4.2) 


2  and  replacing  X^  by  1/0^  In  expression  (2.4.1), 
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HSE2(X) 


1  .  1  .  ,  ,  .2 


which  becomes,  after  algebraic  manipulation: 


MSE2(X) 


Let  ua  compare  *2,  X,  and  X.,  as  estlmatora  of  X2>  Thia 
comparison  will  be  done  for  values  of  (A^ ,  A2)  which  satisfy  the 
assumed  relation  A ^  <  A 2  (or  _>  B^).  Using  this  fact  and  the 

fact  that  MSE(Xj)  ■  Var(X  )  "  rr,  it  is  seen  from  expression 

8  2 

(2.4.2)  that: 


KSE(X2)  >  MSE(A2) 


(2.4.3) 


where  equality  holds  only  in  the  case  where  •  <>2  (equal  means) 

A 

Now  let  us  compare  MSE(X)  and  HSE(A2): 


MSE(A2)  -  MSE(X) 


-  (VV<2V#2)  1  2  2 

x)  .  -L-k - - 1  (“® j  ♦  -  «  0  ) 

2ei(ei4e2)  201®2 


which  after  algebraic  manlpulat ion ,  is  seen  to  be  equal  to: 


fli(ei+le2) 


282(01*V2 


Under  the  restriction  6^,  0^  *  0  this  term  Is  always 
positive.  Combining  this  with  (2.4.3),  we  obtain  the  ordering 

A 

HSE(X^)  >  MSE(X2)  >  MSE(X)  . 

This  holds  for  all  pairs  (X^,  Xp  where  X^  <  X^,  and  conse- 

A 

quently  we  conclude  that  both  X,  and  X^  are  inadmissible  (with 
respect  to  MSE)  In  the  region  of  Interest.  To  Illustrate  this 

A 

Inadmissibility,  Figure  3  gives  a  graph  of  MSE(X^),  MSE(X2>,  and 
MSE(X)  for  a  fixed  value  of  X^  and  values  of  X^  ranging  from 

A 

0  to  X2>  Note  that  Xj  Is  only  a  slight  improvement  over  X^,  while 

A 

X  Is  far  superior  to  either  X,  or  X^. 


cm ) 

Figure  3.  Mean-Square-Error  Comparison  for  the  Case  k  »  2. 
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The  case  where  k  ■  2  and  we  consider  the  estimation  of  A^ 
alone  is  the  only  case  where  an  ordering  relation  among  the  mean- 

square-errors  has  been  proven  (note  we  have  seen  that  MSE(X^) 

2 

-  NSE(A^)  -  A^  for  any  value  of  k  where  A^  ■  A^  •  ***  *  A^) . 
However,  upon  calculation  of  mean-square-errors  for  many  values  of 
k  and  for  various  configurations  of  the  A^  (assuming  A^  <_  •••  <_  A^), 
no  case  has  been  observed  where  the  relation 

A 

MSE(Xk)  >  MSE(Ak)  >  MSE(X) 


did  not  hold.  Consequently,  it  is  hypothesised  that  this  relationship 
is  true  for  all  values  of  k. 

As  an  illustration  we  consider  the  case  k  ■  5,  and  would  like 
to  examine  the  relation  of  the  mean-square-errors  under  vsrlous 
conf lgurst ions  of  the  A^.  To  overcow  the  difficulty  of  graphing  in 
6  dimensions,  we  simplify  matters  by  specifying  a  "constant  improvement 
ratio"  model  where  A  ^  -  r  •  for  1  -  1,  2,  ....  k  -  1  and 
r  ^  1.  When  r  •  1,  we  have  the  boundary  case  of  equal  A^,  and  as 
r  ♦  •  we  have  the  extreme  case  of  very  widely  spaced  A^.  Using  this 
model  and  expressing  MSE  in  terms  of  r  and  A^,  the  previously 
derived  formulas  for  MSE  become: 


MSE(X^) 


,  k-1  .  .2 
(r  A 


r2k*2  A2 
r  Aj 
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and 


As  before,  no  general  closed  formula  for  MSE(A^)  has  been  found. 
Figure  4  gives  a  MSE  comparison  of  the  three  estimates  of  A^  for 
values  of  r  ranging  from  1  to  The  limiting  value  for 

A4  A 

MSE (A.) /MSE- (X)  is  found  by  noting  that  11m  P{ A  -  X.)  -  1  and 

J  5  pMl  5  J 

evaluating  lim  MSE(X,)/MSE(X) .  Dividing  the  MSE  formulas  given 
r— «  5 

above,  we  obtain  (for  general  k): 


11a 

r— 


MSE 


(XV> 


1 


MSE(X) 


-L+(I  - 

2  'k 

k 


1) 


k  -  2k+2 


Mote  from  Figure  4  that  X  has  considerably  better  MSE  than  either 

0K  A 

X^  or  A^  for  all  values  of  r,  and  that  A^  gives  little  improve¬ 
ment  over  X^,  particularly  when  r  Is  large.  This  Is  not  surprising 

A 

since  examination  of  expression  (1.3.3)  shows  that  Afc  will  actually 
be  equal  to  X^  with  Increasing  frequency  as  r  -*  •. 

In  an  attempt  to  determine  why  the  isotonic  estimator  performs 
so  poorly,  the  MSE  is  divided  into  Its  components  of  variance  and 

A 

bias.  Table  1  gives  the  variance  and  expectation  of 
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NSE(X.)/MSF.(X) 


MSK(A.) 


MSE(X) 


Figure  4.  Mean-Square-Error  Comparison  for  the  Case  k  •  5 


for  various  values  of  r.  As  the  table  shows 


1.  is  competitive  with  X  with  respect  to  bias  (becoming  signifi 


respect  to  variance.  Empirically,  X  gives  a  maximum  reduction  in 


variance  over  the  use  of  X,  alone  of  about  20X,  while  X  can  be 


shown  to  reduce  this  variance  as  much  as  96X 


when  r 


Consequently,  even  though  X  is  heavily  biased  for  large  values  of 


r,  its  low  variance  more  than  compensates  to  give  a  reasonable  MSF. 


So  far,  we  have  only  compared  mean-square-errors  for  the 


estimates  of  X.  alone.  Now  consider  total  MSE  over  all  k  of 


Figure  5  gives  a  comparison  similar  to  that  of  Figure  4 


Figure  5.  Comparison  of  Total  MSE  for  Three  Proposed  Estimators 


replacing  final  estimate  MSE'a  by  totals.  For  most  reasonable 
lnproveawnt  ratios,  X  Is  seen  to  have  the  best  total  MSE.  For 
large  Improvement  ratios,  starting  In  the  neighborhood  of  6,  the 
three  estimation  techniques  are  roughly  equivalent. 


2.5.  A  Simple  Modification  of  the  Estimators  of  >. 

A 

The  Initial  performance  of  the  Isotonic  eatlsmtor  in 

the  exponential  case  is  somewhat  disappointing.  We  wish  to  determine 
if  the  situation  can  be  Improved  through  simple  modification  of  the 
proposed  estimators.  Consider  multiplying  an  estimator  by  an 
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appropriately  choaen  conatant  c.  In  general,  if  a  la  aome 

0k 

eat Imator  of  a,  the  value  of  c  which  talnlaizea  MSE(ca)  ia  given 

by 

c. _ . 

®  Var («)  ♦  E(a)2 


Thl*  formula  waa  derived  ualng  elementary  calculua  technique*.  In 
practice  the  formula  may  appear  uaelcaa,  alnce  C*  depend*  on  the 
unknown  parameter  a.  However,  thia  la  not  alwaya  the  caae .  Let  ua 
examine  the  dependence  of  c  on  the  unknown  parametera  X^,  X^,  ••••  * 
In  the  three  caaea  of  lntereat. 

When  X^  la  uaed  to  eatlmale  X^,  we  obtain 


W 


1 

2 


So  c_  la  Independent  of  X  ,  and  we  can  alwaya  aaaure  a  reduction 

\ 

In  MSE  by  dividing  X^  by  2. 

When  X  la  uaed  to  eatimate  1^,  we  obtain 


CX 


16 


Note  that  c-  does  depend  on  the  unknown  X^,  and  consequently  cannot 
be  determined.  In  the  cese  of  equal  X^,  this  doea  reduce  to 
which  Is  Independent  of  the  X^. 

A 

Now  consider  X  .  As  with  X,  c*  will  be  s  function  of  the 

k 

X  .  We  will  compute  c~  for  the  two  "extreme"  cases  of  equal  X 

k  1 

and  widely  spaced  X^.  In  the  equal  X^  case,  we  substitute  the 

sansent  formulas  derived  in  Section  2.3  to  obtain 

c:  -  *(n(k))/(2X2  n(k) )  -  1/2  . 

k 

In  the  case  of  widely  spaced  X ^  (for  example,  consider  the  constant 

A 

ratio  model  with  r  ■*  •),  X^  In  effectively  equal  to  X^,  and  we 
obtain: 


c~  -  C  »  1/2  . 
Xk  *k 


So  we  know  that  c*  Is  equal  to  the  same  value  at  both  "extreme" 

configurations  of  the  1{.  This  leads  to  questioning  the  sensitivity 

of  c*  to  changes  in  the  X  .  Empirical  Investigation  shows  that  c~ 
k  * 

Is  very  Insensitive  to  the  values  of  the  X ^ .  To  Illustrate  this, 

consider  the  familiar  "constant  Improvement  ratio"  model  with  k  •  5. 

Figure  6  shows  a  graph  of  c*  vs.  r.  The  consistency  of  the  results 

k 

lead*  one  to  feel  confident  that  a  nearly  optimal  Improvement  would 

A 

occur  if  X^  were  divided  by  2. 
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Opt  lmal 
Constant 


k  •  5 


Figure  6.  Optimal  Multiplicity  Conatanta 


Not*  that  Figure  6  also  shows  the  relationship  of  c-  to  r. 

In  this  esse,  cj  Is  definitely  sensitive  to  r  (or  to  the  1^). 
Consequently,  we  cannot  confidently  state  that  X  will  be  lsiproved 
by  using  s  particular  ault lpl lest  1 ve  constant  unless  we  know  sore 
about  the  1.. 

Figure  7  Illustrates  the  relationship  among  mean-square-errors 

A 

for  the  estimators  X^/2,  k^/2,  and  X.  X  Is  left  unmodified  due 
to  the  previously  discussed  lack  of  Information  regarding  the  proper 
c-.  As  the  graph  shows. we  have  finally  found  situations  where  an 


Isotonic-based  estlaMte  Is  worth  using  In  the  sense  of  having  reasonable 


k  -  5 


Figure  7.  Relative  Mean-Square-Errors  for  Modified  Estimators 


mean-square-error.  However,  we  do  require  a  large  improvement  In 
mean  lifetime  at  each  stage  (r  must  be  somewhat  larger  than  2 
In  this  example)  In  order  to  realize  the  benefit  of  using  the 
Isotonic  estimate  Instead  of  the  overall  mean  X.  Also,  there  Is 

A 

no  substantial  Improvement  when  using  1^/2  rather  than  X^/2  In 

A 

the  cases  where  ij/2  did  outperform  X. 


2.6.  Asymptotic  Distribution  of  as  k  ♦  ■  (Exponential  Case) 

For  large  values  of  k,  computation  of  the  distribution  of 

A 

smv  be  tlsK-consuming  and  awkward.  Consequently,  It  would  be 
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desirable  to  approximate  this  distribution  by  looking  at  the  limiting 


case,  when  k  *  Let  us  first  consider  the  situation  where  all  of 


the  are  equal  to  a  common  value  1.  We  have  seen  In  Section  2.3 

A 

that  the  distribution  of  Is  given  by: 


PU  <  x)  -  F  (x)  -  1  -  [ 

J-l 


(J-l)! 


where 


6  •  1/1 


For  any  fixed  x  >  0,  note  that  F^(x)  **  strictly  decreasing 

in  k.  Also,  since  Fk(x)  1*  a  distribution  function,  0  <_  F^(x)  _«  1. 
So,  F^(x)  Is  s  bounded  monotone  sequence  In  k(x  fixed)  and 
consequently  a  limiting  distribution,  say,  F_(x),  exists.  Therefore, 

A 

we  can  say  that  1^  converges  weakly  or  In  distribution  to  a  limiting 

A 

random  variable  \m  having  distribution  function  F^(x).  Moments  of 
1  are  found  as  follows: 


E(l“)  -  a  /  ^a-1  fl  -  FJx)]  dx 


«  /  x 
0 


a- 


l“-  }i,  (J-l).  J 


dx 


Since  the  sumiMtlon  Inside  the  integral  la  monotone  In  k,  we  can 
use  a  monotone  convergence  theorem  (cf.  Feller,  Vol.  II,  page  110 
[9j>  to  interchange  limit  and  Integral  signs  and  obtain 


E<0  -  Um 
It— 


/ 

0 


.<*-1 


S 

J-i 


J-2  J-l  -J9 x 

J  (ex)J  e  •* 
(J-l)! 
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The  Integral  given  above  was  evaluated  In  expression  (2.3.7),  and 
substituting  that  result  we  obtain 


In  particular,  with  a  •  1  and  a  -  2  we  have 
E(XJ  "  X  l  1/J  -  An(-)  *  1.644  X 

J-l 

B(X2)  -  212  l  1/J2  -  2X2  n(-)  «  3.288  X2  . 

J-l 

So,  Var(X^)  as  .585  X^  and  HSE(X^)  »  X2.  These  results  Illustrate 
another  difficulty  In  the  use  of  the  Isotonic  estimation  procedure. 

By  the  Kolmogorov  strong  law  of  large  numbers,  X  converges  almost 

A 

surely  to  X  as  k  ■*  “,  and  MSE(X)  ■*  0.  However,  X^  has  the 
unfortunate  characteristic  of  maintaining  a  constant  non-tero  MSE 
while  k  Increases  and  more  Information  about  X  becomes  available. 

The  asymptotic  properties  derived  above  may  seem  to  be  of 
little  value  In  that  the  assumption  of  equal  X^  is  likely  to  be 
unrealistic.  In  general,  of  course,  the  asymptotic  distribution  of 

A 

Xfc  will  depend  on  the  configuration  of  the  X^.  With  our  prototype 
development  model.  It  would  be  reasonable  to  assume  that  we  are 
approaching  some  optimal  level  of  design  as  stagewlse  improvements 
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are  Made.  Consequently,  let  ue  consider  the  case  where  the  A^ 
converge  upward  to  a  Halting  value  A^.  The  following  theorem  allows 
us  to  equate  this  situation  with  the  previously  considered  case  of 
equal  means. 

A 

Theorem  2.4:  The  asymptotic  distribution  of  A^,  the  Isotonic  estimate 
of  the  final  mean  i^.  Is  Identical  In  the  following  two  cases: 

(1)  the  Xj  arc  Independent  exponentially  distributed  random 
variables  having  common  mean  A; 

(2)  the  Xj  are  Independent  exponentially  distributed  random 
variables  having  Individual  means,  where  A  t  A. 

Proof :  Assume  X^  —  exp(A^)  where  A^t  X.  Let  ■  1/ A ^  and 
9  •  1/A .  Let  Yj  ■  (©j/OjX^,  so  that  the  Yj  are  independent  exponentially 
distributed  random  variables  with  coamon  mean  A.  Now  define 

,  Wi 

\  ’  “x(xk’  2  •  *•**  X) 

Wl 

\  -  max(Yk,  —  y  -  - . Y) 

so  ^  snd  sre  the  Isotonic  estimates  based  on  the  X^  and  Yj, 

respectively.  Since  the  Y^  have  common  mean  A,  we  know  Ilk  converges 

A 

In  distribution  to  a  limiting  random  variable  A^  with  distribution 
function 


Therefore,  for  1  >  n  we  have,  replacing 


max  ( b  j ,  hj,  b^)  •  ma  x  ( a  ^ i  ••••  ^  max  (b^  ■  a  ^ ,  ••••  ^  ) 


to  obtain 


v  -  %  <  ^(,t  -  v  (VV‘) ;  ( v»-0 . ,-.i) 


6  Ui 

Now  define  a  •  Since  —  4l,  we  know  that: 


Now  consider  the  Inequality : 


■ax(a^  ♦  bj,  ....  a^  ♦  b^)  <_  aax(a^,  ....  a^)  ♦  aax(b^,  ....  b^) 


with  the  following  values  substituted  for  the  a^  and  b^: 


1 

Using  this  inequality  and  the  fact  that  j  *  1  >  j  -  1  Vi  (since 
the  et  are  decreasing),  we  obtain: 
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Since  0  <_  X.  <  Y  VI,  It  is  easily  seen  (vis  a  term-by-term  comparison) 


that  the  first  term  on  the  right-hand-side  is  less  than  or  equal  to 


aB.  ,  so  we  obtain 


Since  n  is  fixed,  it  is  easily  seen  that  Z 


consequently 


Sow,  using  an  Inequality  similar  to  Chebychev's,  we  have  the  result 


Recall  that  a 


so  we  obtain 


P(  aB 


k 


>  6/2)  <^" 


L 

2  ‘ 


Let  N  -  max(n.m).  By  manipulat 1 ng  probabilities  and  using  results 
given  above  we  obtain,  for  k  ^  N: 


P{Bk  -  ^  >  4}  <  Pta^  +  Zk  >  6)  <  «’U“Bk  >  6/2 J  U  ( Z >  6/2)} 
<  P(aBk  >  6/2)  +  P{ Zk  >  6/2)  <  c/2  +  c/2  -  c  . 


Thus  we  have  shown,  for  given  values  of  6  and  c: 


3N  -  iaax(n,m)  a  k  >  N  *•  P(B.  -  A.  >  6)  <  c  . 

-  k  k 

Therefore,  by  definition,  »k  *  \  £  0  .  Now  recall  Slutsky's  theorem, 
which  states  that  if  6  and  if  -  Z^  ^  0,  then  Z^  (J  W_.  Using  this 

A 

theorem,  we  obtain  the  result  that  *j  Xm  so  and  Bfc  have  the 

same  asymptotic  distribution.  □ 


The  theorem  serves  to  show  that  the  asymptotic  distribution  derived 
for  the  equal  case  is  also  applicable  in  other  situations.  Figure  8 

shows  the  distribution  function  F^fx)  for  various  large  values  of  k. 
Note  that  convergence  to  the  asymptotic  distribution.  F^fx),  is 
extremely  slow.  Consequently,  use  of  an  asymptotic  approximation  for 
finite  but  "large"  values  of  k  is  to  be  approached  with  caution. 
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4 


Taking  all  k  parameters  and  the  assumed  ordering  relation  Into 
consideration,  the  test  can  be  expressed  as  follows: 

Hq  :  Xc  Hj  :  X  c  0, 

whe  re : 

X  *  (ip  ^2»  •••• 

©0  “  {*1*!  -  X2  -  ***  I  xk»  xk  -  aJ 
*  <11*!  I  x2  -  ***  i  V  Xk  <  a}  * 

The  first  approach  considered  was  the  likelihood  ratio  test.  Since 
this  test  Involves  the  maximization  of  a  non-linear  function  over  a 
restricted  space.  It  Is  somewhat  awkward  computationally  and  conse¬ 
quently  we  wish  to  examine  some  simpler  alternatives.  Consider  tests 
based  on  the  three  previously  developed  point  estimates  of 
*k  *  Xk  ’  X*  *n<*  Intuitively  reasonable  tests  would  be  those  which 

A 

reject  HQ  If  (1)  X ^  <  Cjj  (II)  X  <  c2;  or  (III)  <  c^,  for 
appropriately  chosen  constants  c^,  c^.  and  c^. 

A 

To  obtain  a  size  a  test  (used  on  X  ,  c,  must  be  chosen  so 

k  1 


that : 


co«on  value  of  _*  at  which  the  suprenua  rejection  probabilities 
are  attained.  Now  It  la  easily  seen  that  the  ordering: 


Si 


Si 


holds  since  the  Xj  are  non-negat ive .  Letting  c  ■  -  a(ln(l  -  a)), 
the  coaaon  critical  value,  we  obtain  the  following  ordering  aaong 
the  power  functions: 


P.  h 

if  * 


< 


P 

Ml 


< 


i 

ci 


for  any  value  1  c  6^  case  there  Is  no  ’‘best”  test. 

k 

If  A  c  the  test  based  on  .  is  most  desirable  since  It  la 

least  likely  to  falsely  reject  the  null  hypothesis.  However,  If 
A  (  the  test  based  on  X^  la  *o» t  desirable  since  It  Is  Most 

likely  to  correctly  reject  the  null  hypothesis.  The  choice  of  a 
test  In  this  situation  would  depend  on  what  kind  of  error  the  user  Is 
■ost  anxious  to  guard  against  -  that  of  falsely  rejecting  a  valid 
null  hypothesis  or  falsely  accepting  an  Invalid  one.  Figure  9  gives 
exa«ples  of  the  behavior  of  the  three  tests  when  testing 

Hq  :  l j  >  10  vs.  Hj  :  «  10  at  the  a  •  .05  level  when  k  •  3. 

«  k 

Note  that  the  tests  Involving  A.  and  l  X  are  very  biased  -  that  Is, 

J  1  1 

there  are  values  of  A  In  the  ^  region  for  which  the  probability 
of  accepting  Is  greater  than  1  -  a.  This  undesirable 
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characteristic  Is  due  to  the  difficulty  In  determining  anything 
about  Xj  when  only  3  samples,  each  having  large  variance,  are 
available.  The  critical  value  In  this  case  Is  given  by  c  ■  .51, 
and  this  sswill  value  leads  to  high  acceptance  probabilities  for  a 
wide  range  of  values  of  X  In  0L. 

Now  consider  a  similar  hypothesis  testing  problem,  expressed 
from  a  different  viewpoint.  Suppose  we  wish  to  test: 

Kjj  I  X^  «  a  vs.  H1  S  *k  >  *  * 

As  In  the  first  test,  the  hypotheses  can  be  expressed: 

H0  :  V*'  **1  8  —  1  ^ 

where 

%  *  Ul*!  <  —  i  \ .  \ 

^  -  (xlXj  <  •••  <  xfc.  Xk  >  a)  . 

In  this  situation,  the  "burden  of  proof'  would  be  on  showing  that  Xk 
had  exceeded  a  particular  mean.  This  would  be  appropriate,  for 
example,  if  the  device  manufacturers  were  required  to  show  that  their 
newly  developed  prototype  was  superior  to  a  competitive  device  whose 
mean  lifetime  was  known  to  be  a.  To  continue  the  analogy,  the  first 
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tenting  problem  (HQ  :  X^  ^  a  vs.  llj  :  <  a)  presented 

would  be  appropriate  If  the  competitor  were  required  to  prove  that 

the  newly  developed  prototype  wan  inferior  to  hla  own  device. 

Using  the  same  logic  as  in  the  origlncl  testing  situation. 

Intuitively  reasonable  tests  would  be  Chose  which  reject  HQ  If 
„  k 

(I>  Xk  *  cil  (ll>  [  X1  *  c 2’  or  (lII)  Xk  *  c3‘  b*for«. 

Cj,  c,,  and  c^  are  constants  chosen  to  give  the  tests  the  required 
site . 

In  the  first  case,  we  obtain  a  slxe-a  test  If  Cj  In  chosen 
so  that: 


sup  P(  X.  >  c, )  ■  a  . 

%  "  1 

This  rejection  probability  Is  maximized  over  when 

kj  •  Xj  •  •••  ■  X  •  a.  Letting  6^  -  1/a  and  using  the  equal-mean 

distribution  developed  In  Section  2.3,  we  obtain  the  equation: 


where 


1  -  Fk(c1>  •  a  or  *  1  *  ° 


rk<cl)  *  1  “  jij  '  (J-l)I 


So,  It  Is  seen  that  the  critical  value,  c^,  is  equal  to  the  appropriate 
percentage  point  of  the  equal-mean  distribution.  This  distribution 
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depends  only  on  Che  product  °ac|*  which  greatly  simplifies  Its 
tabulation.  Table  2  gives  the  (1  -  a)th  percentage  points  for  various 
values  of  a  and  k.  Note  that  convergence  has  been  reached  by 
k  -  20,  so  the  table  la  appropriate  for  all  values  of  k. 

Suppose,  for  example,  that  k  ■  5  and  we  wish  to  teat 

Hq  1  <  10  vs.  >  10 

using  a  ■  .05.  From  the  table,  the  appropriate  value  for  Ox  is 
3.15.  In  this  application,  0  -  0^  -  1/a  "  .1,  and  x  corresponds 
to  c^.  Consequently,  we  have: 

•lCj  -  3.15  -  C]  -  31.5  . 

at 

Therefore,  the  site  .05  test  based  on  X^  would  reject  HQ  If 

A 

X  >  31.5. 

k 

Now  consider  the  test  based  on  j  X  .  In  this  case,  c.  Is 

l 

chosen  so  that: 

i  ** "  cm’  *  ■ 

Once  again,  the  rejection  probability  Is  maximised  over  0^  when 

Xj  -  ■  X^  ■  a.  For  this  value  of  X ^  has  a  gamma  distribution 

with  parameters  (k,  a).  Consequently,  Cj  Is  seen  to  be  the  (1  -  a)th 
percentile  of  a  C(k,  a)  distribution. 
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For  the  third  test,  based  on  X^,  It  le  easily  seen  that 

should  be  chosen  so  that  P^  (X.  >  c^)  -  a  .  We,  therefore,  obtain: 

k*a 


a 


or 


Cj  ■  -  a(ln  fi)  . 


Unlike  the  first  hypothesis  testing  situation  considered,  we  do 
not  have  a  consistent  ordering  of  power  functions  due  to  the  fact  that 
the  tests  no  longer  have  a  critical  value  In  cossson.  Figure  10  gives 
several  exasples  of  how  the  3  tests  behave  when  testing  HQ  :  X^  <  10 
vs,  J  X^  >  10  when  a  •  .05  and  k  ■  3.  In  Figure  10(a)  HQ  Is 
true  and  use  of  X  Is  seen  to  give  the  highest  probability  of  acceptance 
Unfortunately,  this  high  acceptance  probability  carries  over  to  cases 
where  H(1  Is  false.  Figures  10(b)  and  10(c)  show  situations  where  the 
test  based  on  X  Is  biased.  This  Is  obviously  a  very  undesirable 

A 

trait.  The  tests  based  on  X^  and  X?  are  fairly  close  in  power. 

Xj  tends  to  perfora  better  for  values  In  0^  which  are  fairly  "close" 

A 

to  while  X  does  better  for  values  in  0^  which  are  far  fro® 

0Q.  Again,  the  choice  la  up  to  the  user,  but  tests  based  on  X  are 


not  rec 


nded  due  to  bias. 


ACCEPTANCE  PROBABILITY  ACCEPTANCE  PROBABILITY  ACCEPTANCE  PROBABILITY 


Figure  10.  Comparative  Behavior 


CHAPTER  III 


CONCAVE  ISOTONIC  ESTIMATORS 


3.1.  Concave  Isotonic  Rtgreiilon 

The  results  developed  regarding  the  performance  of  isotonic 
estimators  of  mean  lifetime  in  the  case  of  exponentially  distributed 
X,  are  somewhat  d lsappolnt ing.  In  that  estimates  based  on  the  sample 
mean  X  generally  have  better  MSE.  However,  estimates  which  are 
functions  of  X  have  little  intuitive  appeal  in  that  they  treat  all 
observations  “equally",  ignoring  the  assumption  that  <  ^  <  <_ 

For  example,  early  observations  are  given  Just  as  much  weight  as  more 
recent  ones  in  estimating  X^,  the  final  mean.  This  is  somewhat  counter¬ 
intuitive,  and  consequently  we  wish  to  modify  the  original  model  and 
estimation  process  in  an  attempt  to  find  intuitively  appealing  estimators 
which  perform  better  than  the  isotonic  estimators  . 

One  possibility  would  be  to  specify  a  particular  functional  re¬ 
lationship  among  the  X^  (for  example,  the  X^  might  be  a  linear 
function  of  1  with  non-negative  slope),  and  then  use  the  principle  of 
maximum  likelihood  or  some  other  technique  to  estimate  the  parameters 
Involved  in  the  relation.  This  is  rejected  as  undesirable  since  it 
would  be  very  difficult  in  practice  to  specify  an  appropriate  function 
unless  one  knew  quite  a  bit  about  the  particular  prototype  development 
process  Involved.  However,  a  reasonable  modification  to  the  original 
model  might  be  the  assumption  that,  in  addition  to  an  Improvement  in 
mean  lifetime  at  each  stage,  the  "big"  improvements  occur  early  in  the 
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process  ss  major  design  flaws  are  discovered  and  corrected  while 
later  Improvements  are  limited  to  minor  "fine  tuning"  changes.  This 


could  be  modeled  by  assuming  that  the  A^  are  non-dec reaalng,  as 
before,  but  with  the  additional  provision  that  the  changes  (A  ~  A^) 
are  decreasing.  This  Is  equivalent  to  saying  that  the  A^  are  a 
concave  non-decreasing  function  of  1. 

The  proposed  method  of  estimation,  using  the  modified  model 
described  above.  Is  to  parallel  the  isotonic  regression  concept  by 

*  a  *  2 

finding  values  A  A  ,  ....  A  which  minimize  l  (Y  -  X  )  . 

1  i  *  1-1  1  1 
In  this  case  the  minimum  Is  taken  over  all  values  Y^,  ....  Y^  where 


VY2i 


<  Y. 


*"d  Yl+2  "  Yi+1  -  Yi+1  ”  Yi  for  1  *  2 . .  -  2. 

Recall  that  In  the  case  of  regular  Isotonic  regression  and 

exponentially  distributed  X^,  minimizing  the  sum  of  squared  deviations 
as  given  above  was  seen  to  be  equivalent  to  maximizing  the  likelihood 
function.  This  equivalence  does  not  hold  In  the  "concave  Isotonic" 
case,  so  we  are  no  longer  dealing  with  maximum  likelihood  estimates. 
However,  the  concept  of  least-squares  estimation  Is  a  traditional  one, 
and  would  seem  to  be  especially  appropriate  when  dealing  with  mean- 
square-error. 

The  computation  of  concave  isotonic  estimates  Is  considerably 
more  complex  than  the  regular  isotonic  estimates.  First  consider  the 
simplest  case,  when  k  *  3.  In  this  case,  the  values  for  the  estimates 
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are  easily  found  once  Che  configuration  of  the  X^  is  known.  Table  3 

* 

gives  formulas  for  the  A^  according  to  which  of  seven  regions  in 
3-space  the  vector  (Xj,  Xj,  X^)  falls  into. 

If  k  is  larger  than  3,  the  computations  increase  in  complexity 

a 

and  exact  formulas  for  the  A^  are  not  found.  However,  the  minimization 
problem  Involved  can  be  expressed  in  a  familiar  form  and  the  solution 
determined  using  any  of  a  variety  of  algorithms.  First  note  that: 


k 

I 

l-l 


<Yi- 


V 


- 1 


1-1 


k 

l 


X  Y  + 
i  1 

1-1  1-1 


k 

I 


Since  we  wish  to  minimize  this  expression  over  values  of  the  the 

final  term,  involving  only  X’s,  need  not  be  considered.  Also,  multi¬ 
plication  of  the  entire  function  by  the  positive  constant  1/2  will 
have  no  effect  on  the  solution  vector.  Therefore,  writing  the  function 
and  constraints  in  matrix  form,  the  problem  can  be  expressed  as  follows: 


Minimize:  (1/2)  +  £T  1 

subject  to:  Ay  <  0 


where 
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(2k-3)  *  k 
constraint  matrix 


When  expressed  In  this  form,  the  minimization  problem  is  seen  to 


be  a  classic  quadratic  programming  problem.  Consequently,  any 


quadratic  or  non-linear  programming  algorithm  can  be  used  to  find 


As  In  the  regular  Isotonic  regression  case,  estimator  performance 


has  been  studied  using  Monte  Carlo  techniques.  Results  of  this  study 


are  given  In  Section  3.3 


3.2.  An  Alternative  Concave  Regression  Algorithm 


One  difficulty  with  the  concave  Isotonic  approach  discussed 


In  the  previous  section  Is  the  complexity  of  the  calculations  Involved 


A  potential  user  may  not  have  a  quadratic  programming  algorithm  readily 


available  and,  due  to  difficulties  encountered  In  lesrnlng  or  programming 


one,  may  choose  to  discard  a  potentially  valuable  technique.  Therefore 


an  alternative  method  for  generating  a  set  of  concave  estimates  Is 


suggested.  The  resulting  estimates  do  not  always  minimize  the 


restricted  sum  of  squared  deviations  from  the  original  observation 
(X^),  but  they  do  compare  favorably  with  the  concave  isotonic 
estimates  and  are  much  easier  to  obtain  computationally.  The  steps 
Involved  are  as  follows: 

(1)  Find  the  isotonic  regression,  say  xj,  X1,,  ....  X^ .  of 

the  original  data  X^,  X,,  ....  X^.  Any  of  the  techniques 
given  in  barlow,  et.  al.  could  be  used  here.  This  will 
yield  a  set  of  estimates  which  will  be  non-decreasing 
but  which  may  not  be  concave. 

(2)  Compute  the  differences  dj  -  -  xj  for 

1  "  1,  2,  ....  k  -  1.  We  wish  to  modify  the  d^  to 
Insure  that  they  are  non-increasing  (but  remain  positive). 

(3)  Find  the  antitonic  regression  d|,  . . . ,  d*  j  of 

d^ . d.  .  The  antitonic  regression  is  defined  to  be 

the  set  of  values  dj,  ...,  d*  ^  which  minimize 
k-1 

l  (Y  -  d  )  subject  to  the  constraint  Y.  >  Y  >  •••  ^Y 
1-1  1  1  1 

Algorithms  for  finding  the  isotonic  regression  generally 
have  an  analogous  counterpart  for  finding  the  antitonic 
regression. 

(4)  Use  the  differences  dj  to  generate  a  set  of  estimates. 

A  starting  point,  say  a,  is  required  in  order  to  do  this. 
Choose  a  to  minimize  the  sum  of  squared  deviations  of 
the  estimates  from  the  original  data  values.  The  formula 
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cases.  In  fact,  considering  the  case  k  •  3  and  referring  back  to 
Table  3,  it  can  be  shown  that  the  two  procedures  give  equivalent 
results  except  In  some  Instances  of  case  V. 

As  an  example  of  the  application  of  this  algorithm,  suppose 

that  k  •  5  and  the  observed  lifetimes  are  2,  8,  4,  12,  10. 

Isotonic  and  antitonic  regression  will  be  done  using  the  "pool¬ 
ed  Jacent-violators"  algorithm  given  In  barlow,  Bartholomew,  Bremncr, 
and  Brunk,  page  13  (1).  This  algorithm  simply  pools  anv  adjacent 
pair  of  observations  (and  their  associated  weights)  which  violate 
the  desired  ordering  relation.  This  Is  done  repeatedly  until  the 
ordering  relation  holds,  starting  in  our  case  with  unit  weights.  We 
begin  by  finding  the  Isotonic  regression  of  the  X^. 

Lifetimes  2  8  4  12  10 

Weights  11111 

1st  Pooling  2  6  11 

Weights  122 

After  one  pooling  we  obtain  for  the  Isotonic  regression  the  values 
2,  6,  6,  11,  11.  Consequently,  the  differences  d^  are  given  by 

4,  0,  3,  0.  Since  these  are  not  non- increasing,  we  agsln  pool-adjacent 

violators,  this  time  finding  the  antitonic  regression. 

Differences  4030 

Weights  1111 

let  Pooling  4  2.3  0 

Weights  121 
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One*  again  the  process  terminates  after  only  1  pooling,  and  for 
the  d|  we  obtain  the  values  6,  2.5,  2.5,  0.  Now  compute  the 
and  a: 

o3  -  6.5  +2.5-9 
<*4  -  9  ♦  0  -  9 

-  1.5  . 

Ualng  formula  3.2.1,  we  finally  obtain  the  estimates: 

*j  -  1.5  A*  -  5.5  -  8.0  xj  -  xj  -  10.5  . 

As  In  the  concave  Isotonic  regression  case,  estimator  perfonaance 
(In  the  exponential  case)  has  been  evaluated  via  Monte  Carlo 
techniques.  Results  are  given  In  the  following  section. 

3.3.  Performance  of  Concave  Estimators  (Exponential  Case) 

Computation  of  distribution  functions  and  exact  moments  is 

even  more  difficult  for  the  concave  estimators  than  for  the  regular 

Isotonic  estimators.  Consequently,  evaluation  of  the  performance  of 

•  1 

the  Aj  and  X^  estimators  developed  In  Sections  3.1  and  3.2  has 
been  limited  to  simulation  studies,  and  no  analytic  results  are 
available.  However,  the  studies  consistently  show  the  superiority 
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of  both  types  of  concave  estimators  to  the  original  Isotonic  estimators 

when  the  concavity  assumption  does  Indeed  hold. 

Figure  11  Illustrates  the  results  of  a  typical  simulation  study. 

In  thla  case,  the  number  of  stages  was  set  to  5  and,  as  In  previous 

examples,  the  were  assumed  to  be  exponentially  distributed.  It 

was  assumed  that  the  A^  varied  according  to  the  arbitrary  function 

1 j  •  A  -  B/i  ♦  B/k  (k-5).  Thus,  the  value  for  A  gives  the  final 

mean  of  the  process,  while  B  gives  a  measure  of  the  “spread"  or 

amount  of  Improvement  through  the  development  process.  Mcan-square- 

crrors  are  compared,  using  X  as  a  base,  both  for  estimates  of  the 

final  mean  and  for  total  mran-square-error  over  all  A^. 

As  the  graphs  show,  there  is  a  consistent  ordering  of  mean- 

■quare-errors  over  all  values  of  B.  The  concave  regression  estimates 
a 

Oj)  show  a  very  significant  Improvement  over  the  original  isotonic 

A 

estimates  <  A  ^ ) .  When  considering  the  estimation  of  A^  alone,  the 

a  * 

mean-square-error  for  A^  Is  consistently  less  than  half  that  of  A^. 

The  Improvement  In  total  aean-square-er ror  Is  not  quite  as  dramatic, 
but  Is  also  substantial. 

The  concave  estimates  of  Section  3.2  (a|)  consistently  fall 

a  * 

between  the  A^  and  the  A^.  Although  they  do  not  perform  as  well  as 

a  * 

the  A ^ ,  they  are  also  superior  to  the  A^  while  retaining  a  compu- 

a 

tatlonal  advantage  over  the  A  . 

It  would  appear  that  the  concave  estimators  yield  a  substantial 
improvement  over  the  Isotonic  estimators.  Note,  however,  that  all  NSE 
ratios  illustrated  In  Figure  16  are  larger  than  one.  Consequently,  X 
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Figure  11.  Perforwance  of  Concave  Estimate* 


la  at  1 1 1  the  superior  estimator  (In  terms  of  MSE)  of  the  A^.  The 
choice  of  a  concave  estimator  over  X  would  have  to  be  justified 
by  balancing  the  loss  In  MSE  with  the  concave  estimator's  greater 
Intuitive  appeal.  Note  that  many  of  the  MSE  ratios  are  fairly 
close  to  one.  so  the  loss  In  MSE  may  not  be  significant. 


CHAPTER  IV 

ESTIMATOR  PERFORMANCE  WITH  NON- EXPONENTIAL  LIFETIMES 
AND  RELIABILITY  ESTIMATION 


Isotonic  Estimates  Based  on  Multiple  Samples  per  Staue 


In  previous  sections,  we  have  assumed  that  at  each  stage  a 
single  prototype  with  an  exponentially  distributed  lifetime  is  tested 
Let  us  continue  the  exponential  assumption  but  suppose,  In  addition, 
that,  at  each  development  stage,  several  Identical  prototypes  are 
built  and  tested.  For  simplicity  assume  that  the  same  number  of 
prototypes,  say  n,  are  built  and  tested  at  each  stage.  Let 
Xll*  *12’  Xln  r*Prc"ent  the  observed  lifetimes  at  stage  1. 

Then  the  Joint  density  of  all  observations  can  be  written: 


fx(*ll’ 


*ln . *kl' 


)  -  n  n  f-  e  IJ/  i 

1  l-i  j-i  i 


-f  j 


;  -l. 

.  .  .n  L 

*“  *i 


k  n 

-  H  -L.  e'  Xl 

1-1  x" 


where  x(  Is  the  mean  of  the  observed  lifetimes  at  stage  1. 

Our  goal,  as  usual.  Is  to  efficiently  estimate  the  with 

particular  emphasis  on  X^,  the  final  mean.  Since  the  joint  density 
of  all  the  observations  Is  seen  to  depend  on  the  observations  only 


through  the  X  ,  we  know  from  the  factorization  theorem 

(c.f.  Ferguson,  page  115  |10)),  that  (X  ^ .  ....  X^)  Is  Jointly 

sufficient  for  (X .  X  ).  Consequently,  we  need  only  consider 


estimators  which  are  functions  of  the  X^.  Since  la  easily  seen 

to  have  a  gamma  distribution  with  parameter  vector  (n,  A^/n),  let  us 
simplify  matters  by  assuming  that  at  each  stage  1  a  V5( n,  A^/n) 
random  variable  X^  Is  observed  where  n  Is  known  and  A^  •••  <_  A^. 

As  In  the  previously  examined  exponential  case,  Barlow, 
Bartholomew,  Bremner,  and  Brunk,  page  99  (1972),  show  that  restricted 
likelihood  estimates  for  the  A^  are  given  by  the  Isotonic  regression 
of  the  X^  with  unit  weights  (assuming  an  equal  number  of  samples 
at  each  stage).  We  will  consider  the  same  estimation  procedures 
discussed  earlier  and  focus  attention  on  the  effect  of  multiple  samples 
at  each  stage.  No  exact  distribution  theory  has  been  developed  In  this 
case,  as  the  mathematics  Involved  Is  very  awkward  even  In  the  slmpllat 
case  when  the  A^  are  all  equal.  Consequently,  all  results  given  are 
due  to  simulation  studies. 

As  an  Illustration,  suppose  that  k  ■  5  stages  and  that  the 
means  are  both  Increasing  and  concave,  according  to  the  formula 
A j  •  A  -  B/i  ♦  B/k.  Letting  A  -  10  and  B  ■  8,  we  obtain: 

A  -  3.60  A2  -  7.60  A3  -  8.93  -  9.60  A}  -  10.00 

Figure  12  compares  mean-square-errors  for  various  estimation  procedures 
as  a  function  of  n,  the  number  of  samples  per  stage.  Note  that  as  n 
Increases,  the  performance  of  the  Isotonic  and  concave  Isotonic 
estimators  steadily  Improves  relative  to  that  of  X,  the  overall  mean. 
For  all  three  types  of  isotonic  estimators  considered  there  Is  a 
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threshold  point  where  the  estimate  outperforms  X  for  values  of  n 
beyond  that  point.  This  encouraging  turn  of  events  appears  to  be  due 
to  the  decrease  in  the  variance  of  each  X^  as  n  Increases 
(VarfX^  ■  X^/n).  It  was  seen  In  Section  2.4  (see  Table  1)  that  the 
poor  performance  of  lsotonlc-type  estimators  was  due  to  their  Inability 
to  reduce  variance,  even  though  bias  was  satisfactory.  As  the  variance 
Is  reduced  through  multiple  observations,  bias  becomes  the  more 
significant  contributor  to  mean-square -error  and,  consequently.  It  is 
now  X  that  Is  at  a  disadvantage. 

As  n  Increases,  it  Is  seen  that  the  relative  performance 
of  the  lsotonlc-type  estimators  steadily  Improves.  To  further  examine 
the  effects  of  multiple  samples  per  stage,  let  us  consider  the  asymptotic 
behavior  of  the  various  estimates  as  n  approaches  Infinity  (assume 
that  k,  the  number  of  stages,  remains  fixed).  The  results  will  vary 
according  to  two  possibilities:  (1)  the  1^  are  strictly  Increasing, 
and  (2)  at  least  one  pair  of  the  1^  are  equal.  A  superscript  (n) 
will  be  used  on  the  #(n,  X^/n)  random  variable  X^  and  on  estimates 
of  the  Xt  to  denote  the  dependence  on  n. 

First  consider  the  case  of  strictly  Increasing  X  ,  Since 
X|n)  Is  distributed  as  the  average  of  n  samples  from  an  exponential 
distribution  with  mean  X ^ ,  we  know  from  the  Central  Limit  Theorem  that 

x(n)  -  X 

-W=±  *  N(0.  1) 

as  n  *  •.  The  following  theorem  gives  the  asymtotic  distribution  of 
the  Isotonic  estimates  x|n^  . 
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Then  for  any  1  we  have: 


Theorem  4.1:  Assume  X^  <  X^  <  ***  *  \* 

11a  P{X(n)  -  X.(n))  -  1  . 
n-“  1  1 

Consequently,  xjn^  and  xjn)  have  the  same  asymptotic  distribution: 

>>  -  X 

K  ‘I /■*  *  N(0,  1)  . 

The  theorem  should  be  Intuitively  clear  for  the  following  reason.  If 
the  observed  xjn^  are  properly  ordered,  then  their  isotonic  regression 
Is  simply  the  X^n*  themselves  -  no  changes  need  to  be  made.  Since 
the  means  are  strictly  Increasing  and  the  variances  approach  zero  as 
n  ♦  the  X jn)  are  going  to  be  properly  ordered  with  probability 
approaching  one.  Consequently,  the  xj|n^  will  equal  the  xj°^  with 
ever-increasing  probability.  A  more  rigorous  proof  follows: 

Proof :  Let  e  -  max  (X  -  X  ).  Let  A  denote  the  event  that 
2<i<k  11-1  n 

the  isotonic  regression  Is  equal  to  the  original  observations: 

A  *  (xj*1^  “  X^n*  V  ).  Note  that  A  will  definitely  occur  if  each 
n  1  1  1  n 

xjn)  is  within  e/2  of  X^.  Now, 

P(A„>  >  P(  |x[n)  -  Xj  <  c/2  Vt) 

*  jHj  P{  |x|n^  -  Xj  ^  e/2)  (Independence) 


(Chebychev  Inequality) 


-  1-1 


iSi 


>  n 

-  1-1 


1  - 
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Letting  n  -*  »,  we  obtain:  11a  P(A  )  -  1.  Consequently,  for  any  1 
A  n—  n 

11a  P{X(n)  -  x[n))  -  1. 
n  *-  1  1 


Now  let 


x(n)  -  X  x<n)  -  X 

*1  A.  A  A 

B  -  77--  and  C  -  *  .  __  . 

n  n  ^  ^  fir 


Then,  for  any  positive  6, 


P(|B  -  C  |  <  6)  >  P{ j  B  -  C  |  ■  0)  ■  P(B  -  C  )  . 
n  n  n  n'  n  n 

So,  P {  i  -  C^l  <  6)  -*  1  as  n  ♦  or  B^  -  C^  S  0.  Consequently, 

using  Stutsky's  Theorem  (refer  to  the  proof  in  Section  2.6),  B  and 

n 

C  have  the  saae  asymptotic  distribution.  □ 
n 

Note  that  if  the  X^  are  strictly  increasing  and  strictly 
concave,  an  analogous  proof  could  be  constructed  to  show  that  the 
concave  Isotonic  estlaates  (X*(n)  or  xj*n))  also  have  the  same 


asymptotic  distributions  as  the 

Now  consider  the  second  case,  where  at  least  one  pair  of  the 
A^  are  equal.  In  this  case  we  do  not  have  P(x|n^  -  X^)  converging 
to  one,  and  an  exact  limiting  distribution  is  not  obtained.  However, 
in  theorems  2.1  and  2.2  of  Barlow,  Bartholomew,  Bremner,  and  Brunk 
(1),  It  Is  seen  that  If  A^  £  •••  <_  A^  and  Y^,  •••  Y^n^  are 
a  consistent  set  of  estimates  of  the  A^  then  the  Isotonic  regression 
Y^  of  the  are  also  a  consistent  set  of  estimates.  It  Is  also 

seen  that: 


[<Y[n)#  -  Xt)2  <  [(Yjn)  -  Xt)2  . 


(4.1.1) 


In  our  case  the  may  be  thought  of  as  consistent  estimates  of 

the  A  ,  so  we  can  claim  that  the  X^n  are  also  consistent.  Note 

that  the  above  Inequality  (taking  expectations  on  both  sides)  shows  that 

total  Man-square-error  Is  always  Improved  when  the  Isotonic  estimates 

are  used  rather  than  the  Individual  observations. 

The  proofs  presented  by  Barlow,  Bartholomew,  Bremner,  and 

Brunk  (1]  to  show  Inequality  (4.1.1)  and  consistency  of  isotonic 

estimates  can  be  paralleled  to  achieve  the  same  results  In  the  case 

of  the  concave  Isotonic  estimates  A^n^ .  Consequently,  It  Is 
*/n) 

claimed  that  the  X^  are  also  consistent  estimates  of  the 
(assuming  that  the  X^  are  Indeed  Isotonic  and  concave)  and  that 
total  mean-square-error  Is  always  reduced  when  using  concave  Isotonic 
estimates  rather  than  the  original  observations. 
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4.2.  Estimation  of  Reliability  in  the  Exponential  Case 

In  many  cases  Involving  lifetime  data  the  ultimate  goal  Is 

to  estimate  the  reliability  at  a  given  time  t  (P{X  >  t))  rather 

than  the  mean  lifetime.  For  example,  in  the  prototype  development 

model  we  may  be  interested  in  the  probability  that  the  device  will 

meet  a  predetermined  lifetime  specification.  In  the  case  of 

exponentially  distributed  lifetimes,  the  reliability  at  time  t 

is  given  by  R(t)  ■  e  where  X  represents  the  mean  lifetime 

as  before.  To  proceed  in  the  same  manner  as  when  considering  the 

mean  lifetime  estimation  problem,  we  would  like  to  develop  efficient 

estimators  of  the  parameters  R.(t),  R,(t),  ....  R.  (t)  where 
-t/A  1  1  K 

Rj(t)  *  e  .  The  assumption  that  Rj(t)  <  R,(t)  <  •••  i  V*> 

will  be  made,  analogous  to  the  previous  assumption  of  non-decreasing 

lifetimes.  Note  that  the  time  t  is  assumed  to  be  known  and  fixed. 

Let  us  begin  by  finding  the  restricted  maximum  likelihood 

estimators  of  the  R^ft).  Let 

A  -  {A|A  <  Aj  <  •••  <_  Afc} 

B  -  (R(t) |R1(t)  <  R2(t)  <  •••  <  Rv(t)>. 

/  -t/A  -t/A 

and  define  f  :  A  -»  B  by  f(A^,  ...,A^)-le  ,  e 

The  restricted  maximum  likelihood  estimators  for  the  R^(t)  are  found 
by  maximixing  the  likelihood  function  (reparameterlxed  in  terms  of  the 
Rj(t))  over  the  set  B.  However,  since  f  is  a  1-1  function  from 


)  ’  Uil- 


Ht 


A  to  B,  It  la  aacn  by  the  wtlaua  likelihood  Invariance  property 
(Hood,  Crayblll,  and  Boea,  page  285  (15|)  that  the  reatrlcted  ale's 

A  A 

of  the  R^t)  (or  f(A))  are  given  by  f(A),  where  X  represents  the 

reatrlcted  ale  of  X.  We  know  froa  Section  2.2  that  the  ale  of  X 

la  alaply  the  laotonlc  regression  of  the  observed  llfetlaea 

(or.  In  the  case  of  aultlple  samples  per  stage,  the  Isotonic  regression 

of  the  stage-wise  aaaple  means  ) .  Consequently,  the  restricted 

maximum  likelihood  reliability  estimates  are  found  by  taking  the 

isotonic  regression  of  the  data  and  tranaforalng  according  to  the 

A 

-t/x 

formula  R^t)  •  a 

As  In  previous  studies,  we  wish  to  examine  the  effectiveness 

of  this  estimation  procedure.  Although  no  theoretical  Justification 

Is  presented,  given  the  above  reaults  It  seems  logical  to  also  conalder 

a  _t^*l  1  -  -t/X 

the  estimates  R^(t)  "  e  ,  R * ( t >  •  e  ,  and  R(t)  -  e  *, 

a  i 

where  Aj,  X^,  and  X  are  the  previously  defined  competitive 
estlatatora  of  the  This  study  will  be  limited  to  considering  the 

estimation  of  R^(t),  final  or  current  reliability  In  the  prototype 
development  process. 

Figures  13  snd  14  give  ssmple  results  of  simulation  studies 
made  to  evaluate  estimator  effectiveness.  As  In  previous  examples, 
mean-square-error  Is  used  sa  the  evaluation  criteria,  and  the  results 
are  given  In  terms  of  KSE  rstios  with  the  HSE  of  the  estimator 

8(t)  -  e  In  the  denominator.  Figure  13  gives  results  when  one 

sample  is  tested  st  esch  stage,  while  Figure  14  shows  the  effect  of 
multiple  samples  per  stage. 
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Figure  14.  Relative  MSE  When  Estimating  Reliability 
-Effect  of  Multiple  Samples  per  Stage 
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Not*  that  In  all  figure*  th*  previously  encountered  ordering 
of  mean-square-errora  among  the  three  lsotonlc-based  estimators  Is 
preserved  during  the  transformation  from  mean  lifetime  estimation 
to  reliability  estimation.  However,  a  new  factor  comes  Into  play 
here  -  the  time  t  at  which  the  reliability  la  being  estimated. 

In  all  cases  It  Is  seen  that  the  laotonlc-based  estimators  perform 
better  relative  to  e  l^X  at  earlier  times.  Table  4  gives  a 
typical  breakdown  of  KSE  Into  lta  components  of  variance  and  bias 
at  various  times,  showing  the  effect  of  the  transformation  to 
reliability  estimation  on  both  factors.  Note  In  Figure  13  that 
If  t  la  sufficiently  small  then  the  laotonlc-based  estimates  are 
superior  to  e  As  the  number  of  samples  per  stage  Increases 

(■*•  Figure  14)  and  a  subsequent  reduction  In  variance  occurs,  the 
range  of  t-values  for  which  the  Isotonic  estimator*  are  superior 
to  *  l^X  grows  steadily.  This  la  In  agreement  with  earlier  results 
(Section  4.1)  concerning  the  Increased  effectiveness  of  lsotonic- 
baaed  estimators  when  variance  la  reduced. 

As  th*  nvesber  of  samples  per  stag*  becomes  large,  asymptotic 
results  similar  to  those  of  Section  4.1  can  be  stated.  Letting  X*n) 

I  M 

represent  a  *Jn,  —J  random  variable  (corresponding  to  th*  swan  of 
n  samples  at  stage  1),  it  la  seen  that  xjn)  la  th*  unrestricted  ml* 

-t/X(n) 

of  and  consequently  by  Invariance  e  la  the  mle  of 

-t/lj 

*t(t)  -  *  «•  know  from  maximum  likelihood  theory  (Mood,  Grayblll, 

-t/x<n) 


and  goes,  page  338  (13])  that  e 


Is  asymptotically  normally 


distributed  and  efficient.  Now  It  was  shown  In  Section  4.1  that 

If  the  X  are  strictly  Increasing  then  11a  P( A  f n  ^  ■  x|n^i  *  1. 

1  n*~  *  1 

Consequently,  11a  P,e  •  e  ,  •  1  for  any  t  and  the 

n~  I  I 

Isotonic  reliability  estimators  are  seen  to  have  the  saae  asymp¬ 
totically  normal  distribution  as  the  unrestricted  maximum 
likelihood  estimators.  In  the  case  of  possible  equal  Beans,  only 
consistency  was  shown  In  Section  4.1,  and  this  consistency  will 
still  hold  In  the  case  of  reliability  estimation  due  to  the 
continuity  of  the  function  f(x)  •  e  l^*. 

4.3.  Estimation  for  Normal^  and  Uelbull  Distributed  Lifetimes 

In  previous  studies  we  have  assumed  that  we  were  dealing  with 
exponentially  distributed  component  lifetimes,  yielding  estimators 
based  on  either  exponential  random  variables  (one  sample  per  stage) 
or  gamma  random  variables  (multiple  samples  per  stage).  We  now  wish 
to  determine,  using  simulation  techniques,  whether  the  behavior  of 
the  lstonlc-based  estimators  changes  radically  when  other  comon 
lifetime  distributions  are  assumed. 

Suppose  that  the  prototype  lifetimes  are  approximately 
normally  distributed,  where  for  simplicity  we  will  assume  that  the 
variances  at  each  stage  are  equal  (but  unknown).  This  Is  another 
situation  (recall  Section  1.2)  where  the  restricted  maximum  likelihood 
estimators  of  the  mean  lifetimes  1^  are  given  by  the  Isotonic 
regression  of  the  observed  lifetimes  or.  In  the  case  of  multiple 
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safely  attribute  the  increased  effectiveness  of  the  isotonic 
estimators  to  that  reduction.  Note  that  for  the  normal  case  the 
effect  of  taking  multiple  samples  per  stage  is  not  shown,  since 
this  is  seen  to  be  equivalent  to  having  one  sample  per  stage  with 
an  appropriately  reduced  variance. 

Let  us  consider  the  case  of  Uelbull  distributed  lifetimes 
as  a  second  alternative  to  the  exponential.  The  Uelbull  density  does 
not  fall  into  the  exponential  family  of  densities  for  which  Barlow, 
Batholomew,  Bremner,  and  Brunk  (1)  show  an  equivalence  between 
restricted  maximum  likelihood  estimation  and  isotonic  regression. 
However,  it  may  still  be  of  value  to  consider  and  evaluate  isotonic 
and  concave  isotonic  estimators  based  on  Uelbull  distributed  random 
variables . 

The  Uelbull  density  can  be  expressed  as  follows: 

f(.)  -  *  (!)‘-‘  ,  >  0 

h  n 

where  n  Is  a  scale  parameter  and  0  gives  the  shape  of  the  dis¬ 
tribution.  If  6  ■  1,  the  Uelbull  reduces  to  an  exponential  density. 
If  0  <  1,  the  distribution  has  decreasing  failure  rate  (DFR) ,  if 
0  >  1  it  has  increasing  failure  rate  (IFR).  Since  DFR  distributions 
are  rarely  encountered,  this  study  will  be  limited  to  the  case  where 
0  >  l. 

Figure  16  gives  a  typical  comparison  of  MSE's  for  the  various 
estimators,  showing  the  effect  of  varying  the  shape  parameter  8. 
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Note  that  (1)  the  usual  MSE  ordering  of  the  three  Isotonic-based 
estimators  occurs  again;  and  (2)  the  performance  of  the  isotonic 
based  estimators  improves  relative  to  X  as  ft  Increases.  This 
is,  upon  investigation,  seen  to  be  consistent  with  earlier  findings. 

In  the  simulation  study  illustrated  by  Figure  16,  the  sequence  of 
means  is  held  fixed  over  all  values  of  6  (the  scale 

parameter,  n,  is  varied  accordingly).  If  the  mean  is  held  fixed, 
variance  decreases  as  b  Increases.  This  is  illustrated  in  Table  S. 
Consequently,  we  once  again  encounter  the  situation  of  Improved 
isotonic-based  estimator  performance  (relative  to  X)  when  variance 
is  reduced. 

It  is  assumed  in  the  Welbull  simulations  that  6  does  not 
change  throughout  the  prototype  development  process.  This  assumption 
seems  reasonable  if  we  expect  the  prototypes  to  be  similar  in  design 
and  consequently  exhibit  similar  failure  properties. 

Figure  17  illustrates  the  performance  of  the  various  estimators 
when  g  is  held  fixed  (6  •  2.0)  and  multiple  samples  are  taken  at 
each  stage.  The  estimators  are  then  based  on  the  stage-wise  sample 
means.  Once  again,  the  relative  performance  of  the  isotonic-based 
estimators  steadily  improves  as  variance  decreases. 


CHAPTER  V 


CONCLUSIONS 

The  purpose  of  this  paper  was  to  propose  end  evsluste 
various  techniques  for  estimating  lifelines  or  reliabilities  In 
a  particular  reliability  growth  or  prototype  development  model. 
Evaluation  Involved  several  criteria.  Including  accuracy  (atean- 
square-error) ,  large  sample  properties,  usefulness  In  hypothesis 
testing  situations,  and  Intuitive  appeal.  The  almple  growth  or 
prototype  development  model  and  the  maximum  likelihood  principle 
led  to  the  technique  of  Isotonic  regression,  which  in  turn  led 
to  similar  techniques  Involving  the  additional  assumption  of  con¬ 
cavity.  In  addition,  an  estimate  (X)  was  evaluated  which 
intuitively  Ignored  the  growth  assumption  and  would  perhaps  be 
more  appropriate  In  the  classical  case  of  Identically  distributed 
samples . 

Due  to  computational  difficulties  encountered,  exact  results 
and  Analytical  proofs  were  primarily  limited  to  the  simplest  cases, 
particularly  those  dealing  with  exponentially  distributed  prototype 
lifetimes.  However,  simulation  studies  Involving  more  complex 
situations  lead  one  to  believe  that  the  general  nature  of  the  results 
are  true  for  a  fairly  large  class  of  common  lifetime  distributions. 

The  effectiveness  of  the  lsotonlc-based  estimators  varies,  as 
one  might  Intuitively  suspect,  according  to  several  factors.  If 
there  is  little  significant  improvement  from  stage  to  stage  In  the 
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prototype  development  process,  then  we  are  essentially  In  the  classical 
situation  of  Independent  Identically  distributed  random  variables  and 
there  Is  no  reason  to  believe  an  Isotonlzlng  process  will  Improve 
matters,  which  proves  to  be  true.  In  addition,  the  Isotonic  estimators 
do  relatively  little  to  reduce  variance,  and  consequently  the  overall 
mean  may  still  prove  superior  (in  terms  of  MSE,  not  Intuitive  appeal) 

In  cases  where  there  Is  a  definite  Improvement  In  mean  lifetime  but 
this  improvement  Is  hidden  by  large  sample  variances.  The  situations 
where  a  significant  advantage  Is  gained  through  some  sort  of  Isotonlzlng 
process  tend  to  be  those  where  an  Improvement  does  exist  throughout 
the  process  and  the  variances  of  the  observed  random  variables  are 
"moderate".  Obviously,  on  the  other  end  of  the  scale,  the  Isotonic 
estimators  show  little  Improvement  over  the  use  of  the  observations 
themselves  when  variances  are  very  small  and  the  means  are  strictly 
increasing.  This  is  due  to  the  fact  that  the  Isotonlzlng  process  Is 
essentially  a  rearrangement  of  the  observed  values  Into  the  "correct" 
or  hypothesized  ordering,  and  If  the  variances  are  sufficiently  small 
the  observations  will  be  correctly  ordered  to  begin  with  so  no  changes 
will  take  place.  However,  barlow,  Bartholomew,  Bremner,  and  Brunk 
(1),  do  show  that  if  the  order  assumptions  (and  this  conclusion 
extends  to  concavity  assumptions)  are  correct,  then  In  total  the 
lsotonlzed  values  will  be  at  least  as  good  as  the  originals  with 
respect  to  mean-square-error.  Consequently,  even  with  very  small 
variances  nothing  but  computational  time  will  be  lost,  and  some  gain 
may  occur.  Note  that  a  gain  Is  not  guaranteed  when  considering  the 
estimation  of  any  Individual  parameter. 
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The  only  large  sample  case  Invest  lgat  Ion  Involving  a  large 
number  of  stages,  rather  than  samples  per  atage,  waa  restricted  to 
the  Investigation  of  the  final  Isotonic  estimate  and  the  case  of 
exponentially  distributed  lifetimes.  In  this  case  exact  results 
were  obtainable,  but  were  somewhat  disappointing  In  that  the  usual 
convergence  one  would  hope  to  see  with  Increasing  amounts  of  relevant 
data  does  not  occur.  The  prototype  development  process  may  be  con¬ 
verging  to  an  optimal  design  with  a  corresponding  optimal  mean 
lifetime,  but  the  Isotonic  estimate  of  the  final  mean  In  the  process 
will  not  converge  to  this  optimal  value  as  Its  variance  does  not 
go  to  zero.  This  compares  unfavorable  with  X. 

The  extension  from  the  regular  Isotonic  estimators  to  concave 
Isotonic  estimators  offers  a  significant  Improvement  In  performance 
at  little  expense  In  the  generality  of  the  assumptions.  In  particular, 
variance  reduction  Is  much  better  near  the  final  stage,  and  the 
estimators  have  consistently  proved  superior  to  the  Isotonic  ones  In 
a  wide  variety  of  situations.  The  concave  estimators  proposed  In 
Section  3.2  are  almost  as  good  as  the  concave  regression  estimators 
and  have  a  significant  computational  advantage. 

When  considering  the  estlsutlon  of  reliabilities  rather  than 
mean  lifetimes,  there  Is  no  real  change  In  the  conclusions  regarding 
situations  when  the  isotonic-based  estimators  do  or  do  not  perform 
wall. 
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APPENDIX  A 


Distributions  functions  and  expected  values  of  1^  for  k  *  3 
and  k  •  4.  based  on  exponentially  distributed  X ^ . 
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AlTIN’DIX  b 


This  appendix  contains  proofs  of  the  continuity  of 
*k^l*  ®2*  (Section  2.2)  and  of  l.cana  1,  Section  2.3. 

Thtorts  B:  Let  Xj,  X, . be  Independent  exponentially  dis¬ 
tributed  lifetimes  with  Means  ^  i  ***  1  ^  •  Let 

“  /,  Wl  -  V 

\  “MV  - 2 . . X) 

and 

W  02 . ek)  "  PUk  ^X) 

where  6^  •  1 / X ^  for  1  •  1,  2,  ....  k  and  x  Is  considered  fixed. 
Then 


1  1m 

6  id 
11+1 


VVe; 


wv 


■1 *0 1+1* *1+1 ,0l+2* 


Proof;  Let  Yj,  Y^,  ...»  Yfc  be  lid  exponential  rsndoM  variables 
with  Mean  1,  and  let  Xj  •  Y^/e^  1  -  1,  2,  ....  k.  Then  Xj  has 

an  exponential  distribution  with  wean  1^  •  1/©^  .  Let  ©  ^ 1 ^  . 

be  any  nondecreas tng  sequence  converging  to  6^,  and  define 
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J  -  1,  2,  3,  .. 


be  the  Isotonic  estimate 


X(J)  - 
Xi  A(j)’ 


Let 

k 


of  based  on  X^,  X^.  ...#  X^J  #  X^^,  ••••  X^. 


(•)  (1)  yi  yl 

How  define  X*  •  li»  X:J  •11m  7  '  •  

1  j—  i  y~  e(J)  «1+1 


1*  2' 


x|,)»x^*).  Now  apply  a  monc 
(1968))  to  obtain  the  result 


and  let 

*r 

•  •• 

»!*’• 

A 

that  A 

k 

Is  a 

k  k 

since 

1  (Roy den. 

page  1 

Urn  P(A*J)  <  x) 

J~  k 


-  P(  11m  1,SJ)  <  x) 

J-  k  “ 

-  <  xl  • 


Equivalent ly#  we  have 


11m  a^ ( 0 ^ *  ^2'  ••••  ' 1 

■  We2 . 9i-r  ei+r  ei+i*  ei+2*  ••••  V 


Since  9jJ  Is  an  arbitrary  sequence  converging  to  8^  ,  this 

concludes  the  proof.  □ 


l.en 


l: 


Def lne: 


1  1 

/  /  — 

0  yk/2 


/ 

lk-l/kjy2 


yl  dyl  dy2 


Then 


i  r  kk-2 
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k-l  .  .  .k+r 

y  (-D1  — 

A1  ^-1>J 


...  .i&.W  _L. 

(mn)»  krkJ  J 


Cl) 


where  k  >  2  and  r  >  0. 


Proof:  Evaluate  the  innermost  Integral  of  to  obtain 

ckr)-^^  /  ,m{  ./(km  [i‘(ki1)  yr1ldy2 *•* dyi 

0  yk/2  l(k-2)/(k-l) Jy  L  J 


_jL  L(0)  „  I*=l\  ***  ,  (r*l)l 
r+1  [Ck-1  \  k  )  ck-l  J 


(2) 


This  recursive  relationship  and  the  principle  of  mathematical 
Induction  (on  k)  will  be  used  to  prove  the  result.  Consequently, 
we  need  to  show  that  expression  (1)  holds  for  k  -  2  and  that  it 
satisfies  the  relation  given  In  expression  (2). 

When  k  •  2  we  have,  from  the  definition  of  c£r\ 


/  /  y\  dy.  dy 
0  y  11 


2/2 
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1  y/“]  d’2 

*  A  [‘  -  (i)  m  (£)]  ' 


It  Is  easily  seen  that  the  sane  result  is  obtained  by  letting  k  ■  2 
in  expression  (1).  It  remains  to  show  that  (1)  satisfies  the 
relation  given  in  (2).  Substitution  of  (1)  into  the  right-hand- 
side  of  (2)  gives 
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llL-U 
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l(k-2)!]' 
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l-l 
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[ (k-2) ! J2 


k-2 
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1  (h-l-l)k+r  (r+2) ! 
(k-l-i) !  (r+1+2) ! 


(-1) 


(k-l)rfl(k-l) ! 


Working  with  the  first  term  in  expression  (3),  we  obtain 
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I (k-2) ! )2 
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i-2 


(-1) 


1-1 


(k-l)^"1  1 

(k-i ) !  i ! (k-1) ! 
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(raise  summation  index  by  1) 


.  -L-fV  (-D1-1  _ i _ 

r+1  1  11  (k-D!  1 !  (k-1) ! 


(incorporate  the  first  term  into  the  sum) 


(3) 
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■+l[  Jfl  1  11  J!  (k-j)f(k-l)!  I 
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The  Buonatlon  given  above  la,  by  a  combinatorial  Identity  (Feller, 
Vol .  I,  page  65  18)),  equal  to  xcro.  Therefore,  the  first  term 
In  expression  (3)  reduces  to 


_1 _ k 

r+1 


( (k-1) 1 1 


2 


Raising  the  summation  Index  by  1,  we  obtain  for  the  second  term  In 
expression  (3): 
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TABLE  1 


E(A.)  Var(X)  Var(A.)  MSE(X)  MSE(A  ) 


A,  -  1.00 

k 


TABLE  2 


Percentage  points  for  the  distribution  of  A^  In  the  equal-mean  case 

A 

a  and  (®x)  satisfy  the  relationship  P,(A  *  8x)  •  a,  where  A,  ■ 


25 

1.66 

1.75 

1.79 

1.81 

1.83 

20 

1.86 

1.94 

1.97 

1.99 

2.00 

15 

2.12 

2.19 

2.21 

2.22 

2.23 

10 

2.49 

2.53 

2.55 

2.55 

2.56 

05 

3.12 

3.15 

3.15 

3.15 

3.15 

01 

4.65 

4.65 

4.65 

4.65 

4.65 

a 

k-7 

k-8 

k-10 

k>20 

25 

1.83 

1.84 

1.84 

1.85 

20 

2.00 

2.01 

2.01 

2.01 

15 

2.23 

2.23 

2.23 

2.23 

10 

2.56 

2.56 

2.56 

2.56 

05 

3.15 

3.15 

3.15 

3.15 

01 

4.65 

4.65 

4.65 

4.65 

■  Will— 111  .. 


TABLE  5 


Relating  Variance  to  Shape  Parameter  0  for  Ualbuli  Distribution 

with  Fixed  Means 

k  •  5  stages 


Means  at  each  stage 


3.60 

7.60 

8.93 

9.60 

10.00 

1.00 

12.96 

$7.76 

79.74 

92.16 

100.00 

1.2$ 

8.40 

37.43 

$1.72 

$9.72 

64.80 

1.S0 

$.98 

26.63 

36.79 

42.49 

46.10 

1.7$ 

4.  $1 

20.09 

27.76 

32.06 

34.78 

2.00 

3. $4 

1$. 78 

21.81 

2$. 18 

27.32 

2.2$ 

2.87 

12.77 

17.6$ 

20.38 

22.11 

2.  $0 

2.37 

10. $8 

14.61 

16.88 

18.31 

2.7$ 

2.00 

8.92 

12.32 

14.23 

13.44 

3.00 

1.71 

7.63 

10.  $4 

12.17 

13.20 

SCCUMlTV  Cl  Alt>riC  ATiOM  0>  tHlt  »»0«  <»*>•«  0*>*  lumrfl 


REPORT  DOCUMENTATION  PACE 


ire  MaO  INSTRUCTIONS 

_ BErowr  coMPt.cr.:<c  soru 

OOVT  tCCItllON  MO.  •  atCi»i(MT't  c  AT  AiOC  auwAIM 


•  IlHI  l*M  lUMilti 

PROPERTIES  OF  ISOTONIC  ESTIMATORS  OF  KEAN 
LIFETIME  IN  A  SIMPLE  PROTOTYPE  DEVELOPMENT  MODEL 


Thomas  P.  McWilliams 

•  »(a’0*M>N0  O*0>«lN<IO«  KAMI  AMO  AOOMttt 

Department  of  Operations  Research  and  Department 
of  Statistics  -  Stanford  University, 

Stanford,  California  94305 

•  contnohinc  Otnct  nul  amd  Aooant 
Operations  Research,  Code  434 
Office  of  Naval  Research 

Arlington,  Virginia  22217 _ 


*  tree  or  m*om  a  mmoo  covcaso 
Technical  Report 
s  ecaeeauiMe  oao  aceoar  auMOca 
T~to4f  Aadt  31 WBT  aumltat., - 

N00014-75-C-0561 

~B  aaonaam  lilvtat  Sao/CCT.  laNt 

aaia  a  tow  wait  auuscat 

(NR-047-200) 

________ 

August  16,  1979 
Ti  mumscm  t*  eases 

105 


MoaiToaiMC  toitc*  m  am  c  a  aoomciv»  *tt  oitif)  it  ttcuaiTv  cl  ah.  f i  an  n»«ni 

Unclassified 

It*  OCCl  Altl/iC  ATIOM'  OOtNOaADmS 
iCalOuLS 

OUT  Ml9u TiOM  |T  ATImIn  T  feiftfkle 


APPROVED  FOR  PUBLIC  RELEASE:  DISTRIBUTION  IS  UNLIMITED 


I  17.  DfIT RiBuTiON  ITATImCnT  (•#  lAs  •liffMl  »•  fUfft  M,  1/  #•/»#♦•*!  *•»—«> 


ff  *•«••••*?  ^4  If  **•«*  I 


Mean  lifetime  estlsuitlon 
Concave  Isotonic  regression 


•S.  act  aoaot  fCaorMv*  AM  NWM  { 

Reliability  growth 
Isotonic  regression 
Prototype  development 


M  AltT  ft  AC  ▼  fCMflMH  m  #f  #•  II  Or  w*tl 

Consideration  of  a  simple  prototype  development/rellabll lty  growth  model 
leads  to  observation  of  variables  X^,  X^,  . ...  X^;  where  the  are 

Independent  and  are  assumed  to  have  the  same  distribution  type  with  Individual 
means  1^,  The  problem  Is  to  estimate  the  5^,  which  are  assumed  to  be  non- 
decreaalng  due  to  design  Improvements  In  the  device  under  development.  For 
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many  common  distributions,  estimation  vis  the  restricted  maximum  likelihood 
principle  leads  to  estimates  which  are  the  Isotonic  regression  of  the  Xj 
with  appropriate  weights.  This  paper  finds  the  distribution  of  A  ,  the  mle  of 

00  a  '■  • 

when  the  X^  are  exponentially  distributed.  Afc  Is  then  compared  with  X 
and  X£,  two  competing  estimates  of  A^,  Cm— ,  normal,  and  Welbull  distributed 
X"  are  also  considered,  and  the  usefulness  of  A  Is  seen  to  depend  on  the 

ft 

relative  magnitude  of  the  variances  and  means  of  the  X^.  Results  are  also 
examined  when  the  restricted  mles  are  generated  assuming  the  A^  are  a  non¬ 
decreasing  and  concave  function  of  1. 
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